HomeDocumentationChangelog
HomeRequest DemoContact
HomeRequest DemoContact

Overview

Destinations allow egress of data into a specified data source. This means that Validio can filter out data and store it in a separate table or bucket.

With Destinations, you can analyze the data caught by the filter at a later stage, in a tool of your choice. For example, you can integrate an anomaly table into your data pipelines with automatic jobs to quarantine bad datapoints.

πŸ“˜

Supported Validators

Destinations are currently only supported for the Numeric anomaly and Volume Validator types.

Validio supports a range of destination integrations with different cloud providers, Data Warehouses, and Data streams.

Destination systemDestination type
Data WarehouseBigQuery
Data WarehouseRedshift
Data WarehouseSnowflake
Data streamKinesis

πŸ“˜

Missing Destinations?

We want to make Validio the world’s best Deep Data Observability platform, and would love your feedback. Let us know at [email protected].

Egress datapoints

All egress tables from Validio follow the same schema, regardless of which Source the data is validated from.

πŸ“˜

Validio only writes out datapoints caught by the filter to your destination.

Egress schema

FieldDescription
_validio_uuidAuto generated unique identifier of the anomaly.
_validio_timeTimestamp of when the anomaly was caught.
_validio_is_anomalyTrue for all egressed points.
_validio_applied_filtersWhich filter(s) are applied and whether the filter labeled the datapoint an anomaly.
_validio_datapointThe specific datapoint in questions containing all the field values.

\_validio_applied_filters details

One datapoint can be evaluated by multiple filters, such as on different data fields. As long as one of the filters labels the datapoint as an anomaly, the point is egressed.

The JSON egressed in the _validio_applied_filters lists all applied filters along with a Boolean value that corresponds to whether the filter evaluates the datapoint as an anomaly or not.

You can find the validator_id in the URL when navigating a specific Validator in the platform.

Example

[
  {
    "validator_id": "MTR_ksleKxovmSG4peLApoQK51w",
    "is_anomaly": true
  },
  {
    "validator_id": "MTR_BLCpoSNzGHemp5T32089KhX",
    "is_anomaly": false
  }
]