Root Cause Analysis (RCA)

Learn how Validio RCA identifies causal relationships to help you troubleshoot data quality issues.

Root Cause Analysis

Root Cause Analysis

Validio Root Cause Analysis (RCA) automatically groups incidents that occur on validators around the same time. Using information about the validator and source configurations, the shared fields referenced between incident groups, and upstream or downstream lineage dependencies, Validio identifies causal and correlational relationships among incidents both within individual sources and across sources in the lineage graph. By grouping incidents together, Validio helps minimize the number of alerts that are sent to you.

Validio combines detailed data lineage with data quality monitoring to provide comprehensive analysis that:

  • Explains when an incident occurred
  • Relates incidents to other incidents in your catalog
  • Describes how incidents might affect other catalog assets

You can view this analysis in the Incidents > Root Cause tab.

Prerequisites for RCA

Validio RCA requires no configuration. However, RCA only works on sources that have validators - without validators on the source, Validio cannot create incidents or track incident groups.

How RCA Relationships Work

When an incident occurs on a validator, Validio assigns it to an incident group - a collection of consecutive incidents for that specific validator. The assignment process works as follows:

  • If the previous datapoint processed by the validator was an incident, the current incident joins the same group
  • If the previous datapoint was not an incident, the current incident creates a new group

Validio analyzes information from these incident groups (including source and validator configurations, field-level lineage, dbt manifests, and more) to identify causal and correlation relationships. Because Validio understands how data flows through your system and what types of events can cause alerts, it can determine if one incident group is "caused by" or "correlates with" another incident group.

RCA and Validator Types

While RCA works for all validator types, some validators (such as Freshness and Volume valdidators) can provide more causal context. For more information, see About Validators.

Incident Groups and Shared Validator Fields

Incident groups can occur on different validators when those validators share common elements, such as a field monitored by one validator and referenced in a filter used by another validator. Because of this shared field, incidents are likely to correlate.

RCA Relationship Types

The following table describes the different RCA relationships and provides examples of how Validio identifies them.

RCA RelationshipExample
A related Freshness validator incident in the same source.A row count validator throws an incident at the same time that a freshness validator doesn’t receive data.
A related Row Count validator in the same source.A row count validator that drops significantly and a mean validator tracking a numeric field that shifts at the same time.
A related validator referencing the same field <field>.A numeric validator tracking the mean of a field and another numeric validator tracking the maximum of the same field.
A related validator referencing the upstream field <field>.A validator tracking the mean of total_sales_amount on an upstream source and the mean of total_sales_amount on a source directly downstream.
A related validator referencing the downstream field <field>.A validator tracking the mean of total_sales_amount on an downstream source and the mean of total_sales_amount on a source directly upstream.
A related Freshness validator incident on an upstream source.A freshness validator threw an incident upstream and likely caused the current freshness validator incident.
A related Freshness validator incident on a downstream source.Freshness validators downstream that are throwing incidents as a result of the current freshness validator incident.

Related Resources