HomeDocumentationChangelog
HomeDemoContact
HomeDemoContact

Intro to Destinations

Destinations allow Validio to write out data. This means that Validio can filter out data and store it in a separate table or bucket.

With Destinations, you can analyze anomalous data captured by Validio at a later stage, in a tool of your choice. For example, you can integrate an anomaly table into your data pipelines with automatic jobs to quarantine bad records.

πŸ“˜

Supported Validators

Destinations are currently only supported for the Numeric anomaly and Volume Validator types.

Validio supports a range of destination integrations with different cloud providers, Data Warehouses, and Data streams.

Destination systemDestination type
BigQueryData Warehouse
RedshiftData Warehouse
SnowflakeData Warehouse
KinesisData stream

πŸ“˜

Missing Destinations?

We want to make Validio the world’s best Deep Data Observability platform, and would love your feedback. Let us know at [email protected].

Destination table

All Destination tables from Validio follow the same schema.

πŸ“˜

Validio only writes out records caught by the filter to your destination.

Destination schema

FieldDescription
_validio_uuidAuto generated unique identifier of the anomaly.
_validio_timeTimestamp of when the anomaly was caught.
_validio_is_anomalyTrue for all records.
_validio_applied_filtersWhich filter(s) are applied and whether the filter labeled the record an anomaly.
_validio_datapointThe specific record in questions containing all the field values.

\_validio_applied_filters details

A single record can be evaluated by multiple filters. If any of these filters identify the record as an anomaly, the record will be written out.

The JSON output in the _validio_applied_filters lists all applied filters along with a Boolean value that corresponds to whether the filter evaluates the record as an anomaly or not.

You can find the validator_id in the URL when navigating a specific Validator in the platform.

Example

[
  {
    "validator_id": "MTR_ksleKxovmSG4peLApoQK51w",
    "is_anomaly": true
  },
  {
    "validator_id": "MTR_BLCpoSNzGHemp5T32089KhX",
    "is_anomaly": false
  }
]