HomeDocumentationChangelog
HomeRequest DemoContact
HomeRequest DemoContact

Numeric distribution

Numerical reference statistics between two datasets.

Validator overview

You can use the numeric distribution Validator to make sure that your numeric fields have stable properties over time.

Configuration

StepRequiredParametersOptions
Validator typeNumeric distribution-
ConfigMetricRelative entropy
Mean ratio
Maximum ratio
Minimum ratio
Standard deviation ratio
Sum ratio
ConfigBackfillInitialize with backfill (checkbox)
Source configFieldSelect a valid field from your Source
Source configSegmentation1. Select a configured Segmentation

Or

2. Unsegmented (default)
Source configWindowSelect a configured Window
Source configFilterNo filter (default)
Enum
Null (*1)
String
Threshold filter
Reference source configSourceSpecify a Source to use as reference source
Reference source configFieldSpecify a valid field from reference source
Reference source configWindowSelect a configured Window
Reference source configWindow offset Select how many Windows you want to offset by
Reference source configNumber of WindowsSelect how many Windows to include
Reference source configFilterNo filter (default)
Boolean
Enum
Null (*1)
String
Threshold Filter
ThresholdThreshold typeFixed threshold
Dynamic threshold
Monotonic threshold
Threshold✅(*2)OperatorLess than
Less than or equal
Equal
Not equal
Greater than
Greater than or equal
ThresholdValueNumeric value to validate threshold on

*1 Only applicable for nullable columns.

*2 Only applicable for Fixed thresholds.

Configuration details

Relative entropy

In Validio, relative entropy is based on the Kullback - Leibler divergence measure.

Relative entropy is presented as a percentage where:

  • 0% means identical empirical distributions.
  • 100% means maximal difference in empirical distributions.

📘

You can use relative entropy to validate distribution shifts in your data over time, or to compare the distributions of two data sets.

Ratio metrics

Calculates the ratio of maxmimum, mininum, mean, or standard deviation between the two datasets:

Ratio = target metric/reference metric

Reference source

For information on how you configure the reference source, refer to reference source.