Duplicate Values

Identify duplicate entries in a column to maintain data integrity.

Validator Overview

The Duplicate Values validator works similar to the Volume validator configured for the Duplicate Count or Duplicate Percentage metrics. For more information, see Volume validators.

You can configure volume validators to monitor the following metrics:

Metric Options	Description
Duplicate Count	Validates the number of duplicates.
Unique Percentage	Validates the percentage of duplicates.

Field selection

For the duplicate value metrics, you have to specify one or several fields. Specifying several fields means field values are concatenated before being counted, which is, for example, useful when validating uniqueness of composite keys.

📘
Note
Validio includes NULL values in the counts, unless you apply a NULL filter to remove them.

Metric Configuration Parameters

Parameters	Description	Options
Metric	Select the metric to calculate.	Duplicate count Duplcoate percentage
Field	Select a source field to use for the calculation.	List of available fields with numeric data types.
Filter	(Optional) Use filters to specify which records to include in the calculation.	List of existing filters or create a new filter.
Window	Use windows to define the time-range over which the data is aggregated.	List of existing windows or create a new window.
Segmentation	Use segmentation to break the data into separate groups for analysis.	List of existing segmentations, Unsegmented (default), or create a new segmentation.
Initialize using historic data	Start the validator with historical data to prime the anomaly detection algorithms.

Updated 10 days ago