HomeDocumentationChangelog
HomeRequest DemoContact
HomeRequest DemoContact

Volume

Monitor number of records, non-NULL values, duplicates, unique values etc.

πŸ“˜

Optionally, create a Destination to egress datapoints to a Destination, before you configure a Volume Validator.

Configuration

StepRequiredParameters / Options
Validator typeβœ…Volume-
Configβœ…MetricCount
Percentage
Duplicate count
Duplicate percentage
Unique count
Unique percentage
ConfigDestination1. No Destination (default)

Or

2. Select a pre-configured destination connection
ConfigBackfillInitialize with backfill (checkbox)
Source configβœ…FieldSelect field(s) to monitor
Source configβœ…Segmentation1. Select a configured Segmentation

Or

2. Unsegmented (default)
Source configβœ…WindowSelect a configured Window
Source configFilterNo filter (default)
Boolean
Enum
Null (*1)
String
Threshold filter
Thresholdβœ…Threshold typeFixed threshold
Dynamic threshold
Monotonic threshold
Thresholdβœ…(*2)OperatorLess than
Less than or equal
Equal
Not equal
Greater than
Greater than or equal
Thresholdβœ…ValueNumeric value to threshold on

*1 Only applicable for nullable columns.

*2 Only applicable for Fixed thresholds.

Configuration details

Destination

The Destination parameter allows egress of data to a specified Destination for follow-up inspection. This means that data caught by the filter in this Validator can be stored in a separate table or bucket.

**Volume** Validator Configuration Wizard - Config.

Volume Validator Configuration Wizard - Config.

Field selection

For metrics Count and Percentage you can pick No field (use record) to count, for example, entire rows in a table. You can also pick a specific field to count non-NULL values in that field.

**Volume** Validator Configuration Wizard - Single field in Source config.

Volume Validator Configuration Wizard - Single field in Source config.

For metrics Duplicates count, Duplicates percentage, Unique count and Unique percentage you have to specify one or several fields. Specifying several fields means field values are concatenated before being counted, which is, for example, useful when validating uniqueness of composite keys.

Note: For duplicate- and unique metrics, Validio counts NULL values as any other value, unless you apply a NULL filter.

**Volume** Validator Configuration Wizard - Multiple fields in Source config.

Volume Validator Configuration Wizard - Multiple fields in Source config.