HomeDocumentationChangelog
HomeDemoContact
HomeDemoContact

Volume

Monitor number of records, non-NULL values, duplicates, unique values etc.

📘

Optionally, create a Destination to write out datapoints to a Destination, before you configure a Volume Validator.

Configuration

StepRequiredParameters / Options
Validator type✅Volume-
Config✅MetricCount
Percentage
Duplicate count
Duplicate percentage
Unique count
Unique percentage
ConfigDestination1. No Destination (default)

Or

2. Select a pre-configured destination connection
ConfigBackfillInitialize with backfill (checkbox)
Source config✅FieldNo field (use record) (default)

Or

List of source fields
Source config✅Segmentation1. Select a configured Segmentation

Or

2. Unsegmented (default)
Source config✅WindowSelect a configured Window
Source configFilterNo filter (default)
Boolean
Enum
Null (*1)
String
Threshold filter
Threshold✅Threshold typeFixed threshold
Dynamic threshold
Threshold✅(*2)OperatorLess than
Less than or equal
Equal
Not equal
Greater than
Greater than or equal
Threshold✅(*2)ValueNumeric value to validate threshold on
Threshold✅(*3)SensitivityEnter a numeric value
Threshold✅(*3)Decision bounds typeUpper
Lower
Upper and lower (default)

*1 Only applicable for nullable columns.

*2 Only applicable for Fixed thresholds.

*3 Only applicable for Dynamic thresholds.

Configuration details

Destination

The Destination parameter allows writing out data to a specified Destination for follow-up inspection. This means that data caught by the filter in this Validator can be stored in a separate table or bucket.

**Volume** Validator Configuration Wizard - Config.

Volume Validator Configuration Wizard - Config.

Field selection

For metrics Count and Percentage you can pick No field (use record) to count entire records. You can also pick a specific field to count non-NULL values in that field.

**Volume** Validator Configuration Wizard - Single field in Source config.

Volume Validator Configuration Wizard - Single field in Source config.

For metrics Duplicates count, Duplicates percentage, Unique count and Unique percentage you have to specify one or several fields. Specifying several fields means field values are concatenated before being counted, which is, for example, useful when validating uniqueness of composite keys.

Note: For duplicate- and unique metrics, Validio counts NULL values as any other value, unless you apply a NULL filter.

**Volume** Validator Configuration Wizard - Multiple fields in Source config.

Volume Validator Configuration Wizard - Multiple fields in Source config.

Sensitivity

Higher sensitivity means that the accepted range of values is narrower, which identifies more anomalies. Conversely, lower sensitivity values imply a wider range of accepted values, which identifies fewer anomalies.