Data Warehouse Sources
Connect Validio to your data warehouse to monitor tables and views for data quality issues.
Connect Validio to your data warehouse to monitor tables and views for data quality issues. Validio reads your data incrementally, consolidates queries, and applies warehouse-specific optimizations to minimize cost and performance impact.
Supported Warehouses
Window Types
When defining Windows on data warehouse sources, you can define either a tumbling window or a global window. Generally, when the source includes a timestamp field, we recommend using a Tumbling window. For more information, see About Windows.
Cost and Performance
Validio minimizes warehouse load through several optimizations:
- Incremental processing: Only reads new or changed data since the last poll
- Query pushdown: Performs aggregations and filtering in your warehouse rather than transferring raw data
- Query consolidation: Combines related validations into fewer queries
- Engine-specific tuning: Optimizes query patterns for each warehouse type
Most customers see less than 1% impact on warehouse cost and performance.
Recommendations
To further reduce resource usage:
- Optimize your cursor field: Apply indexing, partitioning, or clustering to the field used for incremental reads
- Prefer tables over views: Querying views or external tables can consume significantly more resources than regular tables
- Consider field size: Large text fields are more resource-intensive to validate than smaller data types
Updated 19 days ago