Reviewing Sources

Review sources on the global Sources page and individual Source details page.

You can review sources on the global Sources page or on an individual Source details page. For an overview of what sources are and how they work, see About Sources. For source management actions (start, stop, reset, and more), see Managing Sources. You can also discover and convert assets into sources from the Catalog.

Sources Page

Global Sources page with data quality tooltip

The global Sources page lists all sources you have access to in Validio. The page includes a Data Quality graph with incidents histogram and a list of sources. You can filter and sort the list, add a new source, or click a source name to navigate to its Source Details Page.

Data Quality Graph

The Data quality over time graph displays the data quality score over the selected time range, a histogram of the severity of the detected incidents, and a count of the incidents for each severity. You can hover over the graph to see more information for each datapoint and bar.

List of Sources

In the source list you see the following information:

Column NameDescription
NameSource name. Click a Source name to navigate to its Source Details Page.
StatusThe current status of the Source, such as Running or Polling. For the full list, see Status.
QualityQuality is a calculated as number of Windows without incidents / total number of Windows.
PriorityThe importance (Critical, High, Medium, Low, None) of the source. Any incidents detected on this source will inherit this priority if the validator does not have a priority.
NamespaceThe namespace assigned to the source.
OwnerThe user responsible for the source.
Latest pollHow long ago data was last polled from the source. Displays an error if the last poll failed.
Latest windowHow long ago data was last seen from the source.
TagsTags assigned to the source. Click the edit icon to add tags.

Status

Possible statuses are:

StatusDescription
InitializedThe Source is created but has never been started.
RunningThe Source is activated and able to process data.
PollingThe Source reads data. This is only allowed from a running state.
IdleThe Source is stopped and is in an idle state.
Pending backfillUser has requested to backfill one or more Validators.
BackfillingThe Source backfills historical records. This is only allowed from a running state.
ResettingThe source is resetting. See Reset for details.

Filtering and Saving Views

You can search by Name, filter by Status, Owner, Tags, and more, and change the Sort column. Once you have a useful combination of filters, you can save it as a View to revisit or share with your organization. For more information, see Filtering and Saving Views.

Source Details Page

Source details Overview tab

To open a source's details page, click on a source name from the Sources Page. The source details are organized into tabs for reviewing validators, segmentations, incidents, and more.

Source Overview

The source overview contains the following information:

NameDescription
DescriptionYou can add a description to the source in Validio. Also automatically imports descriptions from external sources, such as Snowflake or dbt.
Data QualityThe data quality is calculated for each segment and validator on the source as the number of incidents weighted by their severity and divided by the number of windows. For more information, see Data Quality Score.
ReadsThe number of times the source is accessed in the last 30 days.
WritesThe number of times the source is modified in the last 30 days.
Activity LogAll recent operations on the source, including create, update, and delete of resources. Will also display information about when the source was started, stopped, or backfilled with context about the user who did it or if the action was done through an API.

Manage the Source

The management options for individual sources are at the top of the source's details page. The options include start, stop, change ownership and priority, open information panel, mute source, and actions. For the full list of management actions see Managing Sources.

Source Configuration Tabs

All the resources configured on this source are separated into tabs:

Tab NameDescription
ValidatorsReview and manage validators on this source, including their metric results, incidents, and configuration errors.
Schema & ProfilingInspect the source schema, run data profiling, and preview sample rows. Schema change errors are surfaced here.
FiltersCreate and manage filters that specify which records to include when configuring validators and segmentations.
WindowsConfigure the batch size and aggregation period (global or tumbling) used for calculating and validating metrics.
SegmentationsDefine segmentation fields to validate metrics per segment of your data, similar to a GROUP BY in SQL.
IncidentsReview incident groups on this source, assign ownership, and manage incident status.
ErrorsView source errors such as polling failures, schema changes, missing fields, and segmentation limit violations.
ActivityView recent operations on the source, including polling, start/stop events, validator changes, and incident resolutions.