Volume
Monitor number of records, non-NULL values, duplicates, unique values etc.
Optionally, create a Destination to write out datapoints to a Destination, before you configure a Volume Validator.
Configuration
Step | Required | Parameters / Options | |
---|---|---|---|
Validator type | ✅ | Volume | - |
Config | ✅ | Metric | Count Percentage Duplicate count Duplicate percentage Unique count Unique percentage |
Config | Destination | 1. No Destination (default) Or 2. Select a pre-configured destination connection | |
Config | Backfill | Initialize with backfill (checkbox) | |
Source config | ✅ | Field | No field (use record) (default) Or List of source fields |
Source config | ✅ | Segmentation | 1. Select a configured Segmentation Or 2. Unsegmented (default) |
Source config | ✅ | Window | Select a configured Window |
Source config | Filter | No filter (default) Boolean Enum Null (*1) String Threshold filter | |
Threshold | ✅ | Threshold type | Fixed threshold Dynamic threshold |
Threshold | ✅(*2) | Operator | Less than Less than or equal Equal Not equal Greater than Greater than or equal |
Threshold | ✅(*2) | Value | Numeric value to validate threshold on |
Threshold | ✅(*3) | Sensitivity | Enter a numeric value |
Threshold | ✅(*3) | Decision bounds type | Upper Lower Upper and lower (default) |
*1 Only applicable for
nullable columns
.*2 Only applicable for
Fixed thresholds
.*3 Only applicable for
Dynamic thresholds
.
Configuration details
Destination
The Destination parameter allows writing out data to a specified Destination for follow-up inspection. This means that data caught by the filter in this Validator can be stored in a separate table or bucket.

Volume Validator Configuration Wizard - Config.
Field selection
For metrics Count
and Percentage
you can pick No field (use record)
to count entire records. You can also pick a specific field to count non-NULL
values in that field.

Volume Validator Configuration Wizard - Single field in Source config.
For metrics Duplicates count
, Duplicates percentage
, Unique count
and Unique percentage
you have to specify one or several fields. Specifying several fields means field values are concatenated before being counted, which is, for example, useful when validating uniqueness of composite keys.
Note: For duplicate- and unique metrics, Validio counts NULL
values as any other value, unless you apply a NULL filter
.

Volume Validator Configuration Wizard - Multiple fields in Source config.
Sensitivity
Higher sensitivity means that the accepted range of values is narrower, which identifies more anomalies. Conversely, lower sensitivity values imply a wider range of accepted values, which identifies fewer anomalies.
Updated 12 days ago