Configuring Dynamic Thresholds

Dynamic thresholds use smart algorithms to automatically detect anomalies in your data by learning trends, seasonality, and level shifts from historical patterns. Dynamic Thresholds use a combination of smart algorithms to automatically detect anomalies in your data. The threshold model infers trends, seasonality, and peaks, and also adapts to shifts in your data. It learns from historical data and is trained on new data, continuously improving as more data is read.

​

Dynamic threshold algorithm

The dynamic thresholds algorithm delivers precise and reliable anomaly detection that adapts to your data’s unique patterns in real-time, providing immediate value and long-term accuracy. The algorithm provides the following key features: Instant Detection and Resilient Learning The algorithm is designed to provide immediate insights, even on new or backfilled data sources.

• No Manual Training: The algorithms detect anomalies without a manual training phase, even without domain knowledge of the data. For each segment, the first few windows are a burn-in period: Validio calculates the metric and its bounds but does not create incidents. The length of the burn-in period can vary depending on the validator type and configuration. When you start a validator with backfill, the burn-in uses historical windows, so incidents can be detected from the first new window. When you start without backfill, or when a new segment appears later, no incidents are created for that segment until the burn-in period has passed.
• Continuous Improvement: The detection engine is not static; you can provide direct input to refine the model’s accuracy. For more information, see Model feedback and retraining.

Intelligent Level-shift and Seasonality Detection A core mechanic of the algorithm is its ability to distinguish between temporary “noise” and permanent changes in data behavior.

• Trend Adaptation: The algorithm continuously tracks and automatically updates when it detects shifts in seasonality and trends. This makes it ideal for monitoring sources where data is expected to evolve over time. For more information, see Seasonality detection.
• Automatic Recalibration: When a permanent change occurs—such as a feature rollout or configuration updates—the algorithm recognizes the new “normal” and instantly adjusts its baseline, maintaining accuracy without manual intervention.
• Step-like Metrics: Metrics that hold a level and then jump to a new one, known as piecewise constant metrics, are recognized automatically, so routine jumps don’t raise incidents. For more information, see Step-like (piecewise constant) metrics.

Precision Guardrails and Metric Support The algorithm generates adaptive bounds that act as precision guardrails, expanding or contracting based on the data’s current stability.

• Metric Support Estimation: The algorithm includes functionality to estimate the “support” (the range of possible values) of a metric. It uses this to automatically adjust the lower decision bound, ensuring the thresholds make sense for the specific type of data being monitored. For more information, see Metric support.
• Threshold Stability: The algorithm keeps thresholds stable and tight enough to catch genuine incidents even after periods of high variance. It also prevents the bounds from collapsing onto each other for effectively constant or very large metrics so that negligible changes are not flagged. For more information, see Minimum bound width for stable metrics.

Deterministic Freshness Logic For Freshness Validators, the algorithm uses deterministic logic to monitor data arrival patterns.

• Reliable Alerting: By analyzing the exact timing and frequency of data arrivals—including seasonal shifts—the system ensures alerts are consistently tied to genuinely stale data.
• Per-segment Thresholds: The algorithm can monitor data at the segment level, identifying when specific subsets (like a single region) fall behind schedule even if the overall volume appears normal.

For a detailed explanation of how Dynamic Threshold works for freshness, see Dynamic threshold for freshness.

​

Dynamic threshold parameters

The following table lists the parameters for configuring a dynamic threshold. All validator types will have the same configuration options.

​

Sensitivity

Sensitivity defines the accepted range of values for the dynamic threshold.

• Higher sensitivity (lower threshold): Means that the accepted range of values is more narrow, and the model will identify more data quality incidents or anomalies, leading to more alerts. Higher sensitivity is best suited for your most important tables.
• Lower sensitivity (higher threshold): Implies a wider range of accepted values, resulting in fewer incidents and alerts. Lower sensitivity is ideal for less important tables that have historically produced noisy incidents.

Setting the right sensitivity is often an iterative process to find a balance between false positives and alert fatigue versus false negatives and missing real errors. The typical starting sensitivity value for testing is between 2 and 3. The default sensitivity in Validio is 2.0. The following table maps the numeric value of Validio sensitivity presets to standard deviations:

​

Decision bounds

The decision bounds type on the dynamic threshold specifies whether the boundaries for anomaly detection are double or single-sided:

• Lower: Treats only downward deviations as anomalies.
• Upper: Treats only upward deviations as anomalies. For example, this is the default for freshness validators. You do not want to be alerted about too fresh data but rather when your data is late.
• Upper and lower: Detects both upper and lower anomalies.

​

Adaption rate

You can configure how quickly the dynamic threshold adapts to changes in your data. This is controlled by the adaption rate parameter, which offers two settings:

• Fast: The dynamic threshold model responds rapidly to shifts in your data’s behavior. This means that if a new trend or pattern emerges, the bounds within which data is considered “normal” will adjust quickly. This is useful if your data patterns change often.
• Slow: The dynamic threshold model adjusts its bounds more gradually. The model will still adapt to changes, but it will do so over a longer period, giving more weight to historical data. This setting is ideal for more stable data and where the user may not want rapid changes of the bounds but still alert on large outliers.

​

Adaption rate examples

The following example illustrates bounds of dynamic threshold with different adaption rates. Bounds for Fast widen and narrow more rapidly with changes in data, while the bounds for Slow are more persistent in following the downward trend and triggering alarms when the trend changes.

​

Use cases and recommendations

The choice between Fast and Slow adaption depends on the characteristics of your data and your monitoring goals.

​

Model feedback and retraining

You can help improve anomaly detection on dynamic thresholds by giving feedback on both false positives (incorrectly flagged incidents) and false negatives (missed incidents). This feedback is used to retrain the dynamic threshold model for specific segments, making it more precise over time. This targeted retraining helps minimize alert fatigue and ensures the model catches subtle, context-specific anomalies. Dynamic thresholds will only use feedback that shares a similar context to the current data. This means that it will exclude information that does not seem relevant any more. For example, if you are validating your conversion rate and you change the definition inside your warehouse such that the scale of the metric increases by a factor of 100, then DT will not use the model feedback that was previously given to it.

​

False positive feedback: Widening the bounds

When the model incorrectly flags a data point as an anomaly, you can mark it as a False Positive. This feedback tells the model that it was too sensitive in that specific context. To provide this feedback, simply change the triage state of the detected incident to “False Positive”. The model learns from this input and becomes less likely to flag future data points that have a similar value in a similar context. This action effectively widens the decision bounds for that pattern, reducing noise and alert fatigue. For example, as shown in the figure for the “TOTAL_SALES_AMOUNT” Validator, three data points exceeded the dynamic threshold. By marking these incidents as “False Positive,” the model will learn from these cases, adjusting the bounds so that values in a similar context and magnitude are less likely to be flagged in the future.

​

False negative feedback: Tightening the bounds

A false negative occurs when a data point is a genuine anomaly, but the system did not create an incident for it. This can happen if the dynamic threshold bounds have become too wide, perhaps after a period of high variance. To provide this feedback, you can manually select a non incident data point in the graph and mark it as False negative with the toggle. This action tells the model that it was not sensitive enough. The model uses this feedback to tighten its decision bounds more quickly for that specific data pattern, improving its ability to catch similar anomalies in the future. This is especially effective for recalibrating bounds that have “blown up” or become excessively wide. Make sure you only mark data points as False Negative that are true missed anomalies.

​

Reverting feedback

You can revert (undo) both False Positive and False Negative feedback. In the Validator graph, x marks data points for which feedback was given.

• To undo False Negative feedback click on the data point in the graph and use the toggle to switch it back.
• To undo the False Positive feedback change the incident status from False Positive to any other status (including Triage).

​

Feedback and validator reruns

Rerunning a validator recalculates metrics and threshold bounds, which changes how your existing feedback is applied.

• Rerunning the most recent window keeps the feedback you gave on earlier windows. Feedback given on the window being rerun is discarded, so it no longer widens or tightens the recalculated bounds.
• Rerunning from an earlier window rebuilds the dynamic threshold model from the replayed metric history. All existing feedback on that validator is discarded and none of it affects the recalculated bounds.

​

Model feedback versus sensitivity

While both model feedback and sensitivity settings adjust the model’s alerting behavior, they serve different purposes and operate at different scopes. Use sensitivity for broad, initial calibration and model feedback for targeted, ongoing adjustments.

​

Seasonality detection

Dynamic thresholds can automatically adapt to seasonality patterns that appear in your data. You do not have to enable or configure this feature. When there is enough evidence in your data to support the pattern detection, the dynamic threshold will adapt and not trigger an incident if it is caused by the seasonality.

​

Calendric seasonality

Calendric seasonality can relate to business processes and patterns where work may be planned and reviewed in regular cycles (such as weekly, bi-weekly, or monthly) and this behavior is reflected in your data. One example of calendric seasonality is recognizing that a Volume validator returns 0 on all days except the days when the pipeline runs and ingests data.

​

Multiple seasonality patterns

Dynamic thresholds can track more than one seasonal pattern at the same time. For metrics monitored on sub-daily windows (for example, an hourly window), the threshold can learn both the within-day pattern (how values change across the hours of a day) and a day-of-week pattern (how values change across the days of a week). This is useful for metrics that follow a daily rhythm but also vary over the week, such as lower volumes on weekends. Once enough history has accumulated—typically around two weeks of data with sufficient variation—the dynamic threshold recognizes the weekly pattern and adapts to capture these longer seasonalities. The result is tighter, more accurate bounds and fewer false positives on sub-daily metrics that have weekly variation.

​

Metric support

Dynamic thresholds include functionality to estimate the support of the metric by partitioning the sample space (where data can appear) into negative values, zeros, and positive values. Depending on the frequency of the support, the metric gets an estimated positive, non-negative, or unbounded support. The estimated support is not static — it can change over time. Depending on the estimated support, the lower decision bound is adapted:

• If the estimated support is positive, values which are zero or negative are considered incidents.
• If the estimated support is non-negative, negative values are considered incidents.

​

Minimum bound width for stable metrics

Dynamic threshold enforces a minimum bound width so that negligible changes in large or perfectly stable metrics are not flagged as incidents. When a metric is effectively constant, even a small deviation from that value can be flagged as an incident. To avoid unnecessary alerts, dynamic threshold maintains a small band around the metric value. The size of this band scales with the magnitude of the metric. For example, a metric that holds steady around 30 million keeps a band of roughly ±30, so tiny fluctuations pass without triggering incidents while meaningful changes are still caught. This behavior is:

• Automatic: No configuration is required.
• Sensitivity-agnostic: It applies regardless of the sensitivity setting.
• Scale-aware: The minimum band grows and shrinks with the size of the metric’s values.

If you do want to detect very small deviations on an otherwise stable metric, mark a missed data point as a False Negative. This feedback tightens the bounds for that specific pattern and overrides the minimum bound width. For more information, see False negative feedback: Tightening the bounds.

​

Level-shift detection

Dynamic threshold incorporates advanced level-shift detection to identify and adapt to abrupt and persistent changes in your data’s values. A “level shift” often indicates a systemic event, like a configuration change or feature rollout, that causes the data’s baseline to permanently move. This capability ensures you get alerted when such sudden changes occur, and that the anomaly detection remains accurate after the change. This system works by analyzing patterns of consecutive incidents. If a sustained deviation suggests a new stable baseline, statistical tests confirm the shift. Once confirmed, the dynamic threshold intelligently recalibrates by adjusting its training data to this new normal, preventing future false positives based on the old baseline and ensuring continued reliability for evolving data patterns. This adaptive process applies iteratively, ensuring accurate tracking even through multiple sequential level shifts and maintaining reliability as your data evolves. Dynamic threshold also has the ability to make the transition between two levels, or baselines, if it deems that the new level is a reversion to an old one. That is, if dynamic threshold has recently seen one or more level shifts and there is a new incident group, it will try to connect the new incidents with the levels that was previously seen. Common scenarios where Level-shift Detection is useful:

• Step changes in metrics: Handles temporary shifts and reversions without over-alerting. For metrics that move in steps as a matter of routine, see Step-like (piecewise constant) metrics.
• New baselines after changes: Recognizes stable new levels after feature rollouts or configuration updates
• Recurring calendar-based shifts: Detects predictable changes tied to the calendar, including weekly recurrence (for example, a regular weekend dip) and date-based shifts (for example, the 1st of the month). Recognizing weekly-recurring patterns lets the dynamic threshold adapt faster to anomalies that repeat on a weekly cycle.
• Variance changes: Identifies when data flatlines, becomes unusually noisy, or shows other significant changes in behavior

​

Step-like (piecewise constant) metrics

Some metrics move in steps: they hold the same value for several windows, then jump to a new level and hold that one. The technical term for this is a piecewise constant metric. Examples include a count of active configurations, a maximum value that only changes when a new batch lands, or a price that changes occasionally. On these metrics, each jump looks like a sudden change, so without special handling every routine jump could raise an incident. Dynamic thresholds recognize step-like metrics automatically and adapt to them. You do not have to enable or configure this feature.

• Routine steps are tolerated: The dynamic threshold learns how large the metric’s steps usually are and widens the bounds to cover a step of that size, in either direction. It also keeps the current level inside the bounds while the metric holds it.
• Unusual steps still raise incidents: A step much larger than the routine ones raises an incident. If the metric stays at the new level, the dynamic threshold treats it as a level shift and recalibrates to it.
• Sensitivity still applies: A higher sensitivity tolerates smaller steps, and a lower sensitivity tolerates larger ones. Model feedback works as it does for any other metric.

Step-like handling turns on when a metric has at least 30 data points and most of its recent values sit on flat levels, with several steps between them. It turns off by itself if the metric stops moving in steps, for example when its values start to vary from window to window. When it turns on, the metric graph shows a model change indicator with the explanation “Dynamic Threshold found a new signal in the data and adapted accordingly.”

​

Model change indicators on the metric graph

The metric graph marks the moments when the dynamic threshold model re-bases itself. When the model retrains or recalibrates—for example after a level shift or when it detects a new pattern—a vertical dashed line appears on the graph spanning its full height at the timestamp of that window. This helps you see exactly when, and why, the bounds shifted. To see why a model change occurred, hover or click the datapoint at the dashed line. The tooltip shows a Model changed section at the top, which includes:

• A Detected at timestamp for the model change.
• A short explanation of why the model changed.
• The threshold bounds from before the change, so you can compare them with the new bounds. The regular Upper and Lower bound rows below reflect the updated bounds after the change.

A model change can happen for the following reasons: