Skip to main content
Validio automatically derives a schema for every source, so your data validation rules are built on accurate schema information without manual setup. It uses three approaches:
  • Schema from metadata: For most structured data types, Validio reads the schema from the data source metadata. For example, reading from INFORMATION_SCHEMA in a data warehouse.
  • Schema from inference: When a predefined schema doesn’t exist, such as for JSON or other semi-structured data types, Validio infers the schema from the existing data patterns.
  • Manual configuration: Depending on the data source and types, you can manually configure the schema by selecting fields, including nested fields, for validation.
The source schema list showing detected fields and their profiling results

Source schema list with profiling results

​
Supported complex data types

In addition to structured data, Validio supports semi-structured and other complex data types. You can select these fields or nested fields when you configure a source.

​
Data warehouses

​
Data streams

JSONPath expressions and arraysCurrently Validio does not support data validation within an array. However, you can validate the size of an array.Validio uses JSONPath expressions to represent data structures. For each array, Validio adds a computed numeric field named some_array.length() to represent the size of an array. For more information, see Semi-structured Data Syntax.

​
Understanding nullable fields

If the automatically inferred schema doesn’t match your expectations for incoming data, you can modify the nullability settings and data types to better reflect your data structure.
  • Check the Nullable option to include datapoints with NULL values in validation.
  • When unchecked, datapoints with NULL values in that field will be excluded from validator metrics.
This gives you control over how missing or null data affects your validation results.

​
When automatic detection fails

A schema detection error offering the option to upload a sample data file

Schema detection error with option to Upload sample data file.

Schema inference may encounter issues in these scenarios:
  • Large tables: Timeout occurs when source tables are too large to process efficiently
  • Unknown data types: Cannot determine appropriate data types for schema fields
  • Mixed data types: Semi-structured data with inconsistent data types across rows

​
Upload a JSON sample

When automatic detection fails, upload a JSON sample file to help Validio understand your schema structure. Sample data format:
JSON
Case Sensitivity: The properties (fields and values) used in the uploaded sample data file must match the case conventions of your data source. For example, Snowflake defaults to uppercase, so your sample data should use uppercase field names.

​
Schema change validation

Validio automatically validates schema changes for structured data in data warehouses and object storage files. Validio runs schema checks hourly, and reports detected changes as source errors, not incidents. You can review them on the source’s Schema tab, and receive notifications for them with a notification rule of the Schema change type. For more information, see About Source Errors and Adding Notification Rules.