Metadata
Validate Freshness or Row Count using warehouse metadata.
Validator Overview
For data warehouse sources that provide information as metadata, you can add metadata validators for Freshness or Row Count. Metadata validators do not monitor the actual data in the warehouse. Instead these validators check against the warehouse metadata, allowing you put more validators on your sources without increasing the performance impacts.
Important
Metadata validators are only available for Snowflake and BigQuery warehouses.
You can configure the following metadata validators:
| Validator Type | Description |
|---|---|
| Freshness (metadata) | Ensure data is timely by checking the last modification time via warehouse metadata. |
| Row Count (metadata) | Verify the number of rows by checking data volume via warehouse metadata. |
Configuration Details
When configuring metadata validators, you will only need to select the validator type, define the threshold, and add validator details. Metadata validators can only use global windows and do not support segmentation, filters, or backfill of historical data.
Global polling schedule for metadata validators
The polling schedule configured on the source does not apply to metadata validators added to that source. Instead, all metadata validators across sources run on a one hour polling schedule.
Updated 22 days ago