Intro to Destinations
Destinations allow Validio to write out data. This means that Validio can filter out data and store it in a separate table or bucket.
With Destinations, you can analyze anomalous data captured by Validio at a later stage, in a tool of your choice. For example, you can integrate an anomaly table into your data pipelines with automatic jobs to quarantine bad records.
Supported Validators
Destinations are currently only supported for the Numeric anomaly and Volume Validator types.
Validio supports a range of destination integrations with different cloud providers, Data Warehouses, and Data streams.
Missing Destinations?
We want to make Validio the world’s best Deep Data Observability platform, and would love your feedback. Let us know at [email protected].
Destination table
All Destination tables from Validio follow the same schema.
Validio only writes out records caught by the filter to your destination.
Destination schema
| Field | Description |
|---|---|
| _validio_uuid | Auto generated unique identifier of the anomaly. |
| _validio_time | Timestamp of when the anomaly was caught. |
| _validio_is_anomaly | True for all records. |
| _validio_applied_filters | Which filter(s) are applied and whether the filter labeled the record an anomaly. |
| _validio_datapoint | The specific record in questions containing all the field values. |
\_validio_applied_filters details
\_validio_applied_filters detailsA single record can be evaluated by multiple filters. If any of these filters identify the record as an anomaly, the record will be written out.
The JSON output in the _validio_applied_filters lists all applied filters along with a Boolean value that corresponds to whether the filter evaluates the record as an anomaly or not.
You can find the validator_id in the URL when navigating a specific Validator in the platform.
Example
[
{
"validator_id": "MTR_ksleKxovmSG4peLApoQK51w",
"is_anomaly": true
},
{
"validator_id": "MTR_BLCpoSNzGHemp5T32089KhX",
"is_anomaly": false
}
]
Updated 12 days ago