HomeDocumentationChangelog
HomeRequest DemoContact
HomeRequest DemoContact


Configure a Source to authenticate Validio and define a data source to validate. Supported source types include the Data Warehouses, Object storage, and Data streams listed in this table:

Source systemSource type
[BigQuery](🔗)[Data Warehouse](🔗)
[PostgreSQL](🔗)[Data Warehouse](🔗)
[Redshift](🔗)[Data Warehouse](🔗)
[Snowflake](🔗)[Data Warehouse](🔗)
[GCS](🔗)[Object storage](🔗)
[S3](🔗)[Object storage](🔗)
[Kinesis](🔗)[Stream](🔗)
[Pub/Sub](🔗)[Stream](🔗)
[Pub/Sub Lite](🔗)[Stream](🔗)

Missing Sources?

We want to make Validio the world’s best Deep Data Observability platform, and would love your feedback on what Sources to support. Let us know at [[email protected]](🔗).

# Schema

Validio defines a schema for every source, either based on metadata or inference.

**Schema from metadata:**

Validio reads the schema from the metadata in the data source, for example, from `INFORMATION_SCHEMA` in a Data Warehouse. This is true for most structured data types.

**Schema from inference:**

Validio infers the schema from the existing data when no pre-defined schema exists. This is true for most semi-structured data types, for example `JSON`. Depending on your Source type, it might take a few seconds to infer the schema.



Validio can only infer schema when data exists in the source.

## Detect schema changes

Validio will automatically detect schema changes for structured data types in Data warehouses and files in Object storage.

You can get notified about schema changes through [Notifications](🔗) and in the [Activity pane](🔗).

# Semi-structured data

In addition to structured data, Validio supports semi-structured and other complex data types. You can select these fields or certain nested fields when you [configure a Source](🔗).

JSONPath

Validio uses JSONPath to represent data structures.

For example, the JSONPath expression `some_array.length()` represents the size of an array.

## Supported semi-structured and complex data types

Source systemData type
BigQueryJSON, RECORD (NULLABLE and REPEATED)
PostgreSQLJSON, JSONB, array types
RedshiftSUPER
SnowflakeARRAY, OBJECT, VARIANT
KinesisJSON
Pub/SubJSON, protobuf, Avro
Pub/Sub LiteJSON, protobuf, Avro

Array support

Currently, Validio does not support data validation within an array.

You can validate the size of an array. For each array, Validio adds a computed field named `some_array.length()`, which you can validate as a numeric field.