HomeDocumentationChangelog
HomeRequest DemoContact
HomeRequest DemoContact

Numeric anomaly

Identify numeric anomalies in your data with Machine Learning algorithms.

πŸ“˜

Anomaly egress

Optionally, create a Destination to egress datapoints to a Destination, before you configure the numeric anomaly Validator.

Validator overview

Validate individual field values for every datapoint Validio reads by comparing the field value in a reference source. Dynamic anomaly bounds are configured with the sensitivity parameter.

The Numeric anomaly validator identifies anomalies based on either count or percentage:

  • Count: Counting how many datapoints are identified as an anomaly in each window.
  • Percentage: Counting the share of datapoints that are identified as an anomaly in each window.

Configuration

StepRequiredParametersOptions
Validator typeβœ…Numeric anomaly-
Configβœ…MetricCount
Percentage
Configβœ…SensitivitySelect a numeric value
ConfigAdvanced configMinimum absolute difference
Minimum number of reference datapoints
Minimum relative difference percent
ConfigDestination1. No Destination (default)

Or

2. Select a pre-configured Destination
ConfigBackfillInitialize with backfill (checkbox)
Source fieldsβœ…Select source fields1. Specify from list of source fields with numeric data types

Or

2. Select all option (checkbox)
Source configβœ…Segmentation1. Select a configured Segmentation

Or

2. Unsegmented (default)
Source configβœ…WindowSelect a configured Window
Source configFilterNo filter (default)
Boolean
Enum
Null (*1)
String
Threshold filter
Reference source configβœ…SourceSelect a Source to use as reference source
Reference source configβœ…FieldSelect a valid field from your reference source
Reference source configβœ…WindowSelect a configured Window
Reference source configβœ…Window offsetSelect how many Windows you want to offset by
Reference source configβœ…Number of WindowsSelect how many Windows to include
Reference source configFilterNo filter (default)
Enum
Null (*1)
String
Threshold Filter
Thresholdβœ…Threshold typeFixed threshold
Dynamic threshold
Monotonic threshold
Thresholdβœ…(*2)OperatorLess than
Less than or equal
Equal
Not equal
Greater than
Greater than or equal
Thresholdβœ…ValueNumeric value to validate threshold on

*1 Only applicable for nullable columns.

*2 Only applicable for Fixed thresholds.

Configuration details

Sensitivity

Higher sensitivity means that the accepted range of values is narrower, which identifies more anomalies. Conversely, lower sensitivity values imply a wider range of accepted values, which identifies fewer anomalies.

Destination

The Destination parameter allows egress of data to a specified Destination for follow-up inspection. This means that data caught by the filter in this Validator can be stored in a separate table or bucket.

Advanced config

Minimum absolute difference:

The minimum absolute difference between the field value and the mean of the reference distribution for the point to be considered an anomaly.

For example, if set to 10, the difference between the mean of the reference distribution and the datapoint being validated must be greater than 10, and be outside the dynamic bounds to be considered an anomaly. Essentially, this is an ignore any incidents within the difference parameter.

Minimum number of reference datapoints:

Minimum number of datapoints in reference source before triggering a metric calculation.

Minimum relative difference percent:

Minimum difference for points to be considered an anomaly expressed in relative terms, divides absolute difference with absolute of the mean of the reference data.

For example, if the mean of the reference distribution is 10, and user sets 10% as parameter value, then, data points falling between 9 and 11 are not considered anomalies.

We recommend that you use this option instead of minimum absolute difference, when you're more interested in the relative difference to the reference mean, than the absolute difference.

**Numeric anomaly** Validator Configuration Wizard - Config.

Numeric anomaly Validator Configuration Wizard - Config.

Reference source

For information on how you configure the reference source, refer to reference source.