> ## Documentation Index
> Fetch the complete documentation index at: https://docs.validio.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Monitoring and Validation

> Set up monitoring and validation on a data warehouse source by adding a credential, creating a source, and configuring a validator.

This is the first part of the tutorial. It shows you how to quickly set up monitoring and validation on a data warehouse source: create a credential to connect to the warehouse, specify the data source, and add a validator to track a metric on the source. Make sure you have completed the tutorial [prerequisites](https://docs.validio.io/v11.0/docs/tutorials/getting-started-tutorial.md#prerequisites) before you start.

This tutorial follows a Google BigQuery table that summarizes sales data, with hourly summaries of sales by product and customer demographics, including total sales amount, quantity, and average discount. You can follow along with a similar table of your own — the steps are the same for any supported source.

> [!NOTE]
>
> The tutorial steps are specific to Google BigQuery, but the setup flow is the same for all source types. You should be able to complete these steps to monitor and validate data from any supported data warehouse that you can access.

## Add a data warehouse credential

The first step to monitoring is adding a credential to connect Validio to the source.

![The Credentials page listing the Tutorial (GCP) credential](https://mintcdn.com/validio/36lh-6_uXUsulPvT/images/docs/tutorials/tutorial-credentials.png?fit=max\&auto=format\&n=36lh-6_uXUsulPvT\&q=85\&s=b17091a71add947e0409e538521b23b6)

Credentials page with GCP credential, Tutorial (GCP)

Since the Tutorial data is in a Google BigQuery warehouse, you need to add a credential for Google Cloud Platform (GCP). The GCP credential connects Validio to a service account that provides access to the relevant source tables in the warehouse.

> [!NOTE]
>
> If you have data in another warehouse, you can refer to our integration guides for supported warehouse sources to find instructions for adding the relevant credential. See [Data Warehouse Sources](https://docs.validio.io/v11.0/docs/integrations/data-warehouses.md) in our product documentation.

To add a Google Cloud Platform (GCP) credential:

1. Navigate to **Credentials** and click **+ New credential**.
2. Select the **Namespace** where the resource will be created.
3. For **Credential type**, select Google Cloud Platform (GCP).
4. Under **Configuration**,
   1. Enter a unique **Name** for the credential.
   2. Paste or upload the contents of a **Service account JSON-file** that provides the appropriate roles and permissions to access the relevant tables in the warehouse.
   3. (Recommended) Click **Test credential** to verify the connection to the GCP service account. If the test fails, check your configuration settings.
5. (Required for this tutorial) Check **Use for catalog and schema checks** to discover assets from the credential and add the assets to the Catalog page.
6. Click **Create credential**.

After the credential is created, Validio connects to the service account and collects information, such as metadata and lineage, to populate the Catalog page. You can navigate to the Catalog page to view all the discovered assets associated with the credential.

> [!NOTE]
>
> For more information about credential types and best practices, see [About Credentials](https://docs.validio.io/v11.0/docs/credentials.md) in our product documentation.

## Convert a catalog asset into a source

Although you can view Catalog information about the assets discovered on the credential, Validio does not read data or poll for updates from the assets. To start monitoring data, you need to add a data source, which you can quickly do by converting a catalog asset into a source.

![The Catalog page listing table assets discovered from the tutorial credential](https://mintcdn.com/validio/36lh-6_uXUsulPvT/images/docs/tutorials/tutorial-catalog-table-assets.png?fit=max\&auto=format\&n=36lh-6_uXUsulPvT\&q=85\&s=50a4ac78a2eebbb5eb309abf3f729329)

Catalog page showing Tables discovered on the Tutorial (GCP) credential.

The data used in this tutorial consists of tables for simulated sales and customer data. For this part of the tutorial, you will select one table asset, *gold\_\_sales\_summary*, and convert it into a source:

1. Navigate to **Catalog > Table assets** to view a list of all assets discovered from this credential.
2. Search for and click **gold\_\_sales\_summary** to open its details page.
3. Click **Convert to source** to open the "Convert catalog assets to source" configuration.
4. Configure the **Catalog asset**:

   1. **Credential** - Select the tutorial credential you created to associate with the asset when converting to a source.
   2. **Polling schedule** - Use cron expression to specify how frequently to check for changes on the source.
   3. Review the configuration summary, and click **Continue**.

![The convert catalog asset to source dialog with the credential and polling schedule](https://mintcdn.com/validio/36lh-6_uXUsulPvT/images/docs/tutorials/tutorial-asset-config.png?fit=max\&auto=format\&n=36lh-6_uXUsulPvT\&q=85\&s=e7b4068ebd5bbb8d0582a929a2cb4eb4)

Catalog asset to source configuration

5. Under "Validators", you have the option to add metadata validators to the source. Click **Continue** to skip this step.
6. Click **Create source** to finish the configuration.

You can click **View source** to open its details page where information about the source and its configuration is organized into tabs. Even though the source is created, monitoring does not happen until you **Start** the source. Before you start the source, you will add a Validator to define a metric to track and validate data.

![The Overview tab of the gold\_\_sales\_summary source](https://mintcdn.com/validio/36lh-6_uXUsulPvT/images/docs/tutorials/tutorial-source-details.png?fit=max\&auto=format\&n=36lh-6_uXUsulPvT\&q=85\&s=8f37700b7646afb292a9f58107a328d5)

gold\_\_sales\_summary Overview tab

> [!NOTE]
>
> For more information, see [About Data Catalog](https://docs.validio.io/v11.0/docs/catalog.md) and [Catalog Assets](https://docs.validio.io/v11.0/docs/catalog/catalog-assets.md) in our product documentation.
>
> If the data source you want to add is not in the Catalog, you can also directly add sources using the **Sources** page. For more information, see [About Sources](https://docs.validio.io/v11.0/docs/sources.md) in our product documentation.

## Add validation to the source

Now that you have a source, you will configure a Numeric Statistics validator to track the mean of the total sales amount field. For time-series data like this, tracking the daily **Mean** over a **tumbling window** with a **dynamic threshold** lets Validio learn the normal pattern and flag days where sales deviate — without you having to set fixed limits by hand.

![The Validators tab of the source with no validators configured and the source not started](https://mintcdn.com/validio/36lh-6_uXUsulPvT/images/docs/tutorials/tutorial-source-validators-tab.png?fit=max\&auto=format\&n=36lh-6_uXUsulPvT\&q=85\&s=ea57e8055d886a4d515e10072fc729ea)

Source Validators tab, no validators and not started

To add a Numeric Statistics validator,

1. Navigate to **Sources > gold\_\_sales\_summary** and click **+ New validator**.
2. Under **Validator Type**, select **Metrics & Validity > Numeric Statistics**.
3. Under **Metric**,
   1. Configure the metric to calculate the **Mean** using the field *total\_sales\_amount*.
   2. Select or create a **Tumbling window**, to aggregate the calculation over daily non-overlapping time intervals, **Every 1 day on "timestamp"** with "No slide config".

      ![The new tumbling window dialog configured for every 1 day on the timestamp field](https://mintcdn.com/validio/36lh-6_uXUsulPvT/images/docs/tutorials/tutorial-tumbling-window-config.png?fit=max\&auto=format\&n=36lh-6_uXUsulPvT\&q=85\&s=b0aad9922b473a17feef3a8168957b92)

      New Tumbling window configuration

      > [!NOTE]
      >
      > Because *gold\_\_sales\_summary* is a data warehouse source with time-series data, we recommend using a tumbling window to group the data for validation. Global windows, which are automatically created when you add a source, are better suited for dimensional tables. For more information about configuring windows, see [About Windows](https://docs.validio.io/v11.0/docs/windows.md) in our product documentation.
   3. Check "Initialize with historic data". The amount of historic data is configured with the tumbling window.
4. Under **Threshold**, configure a Dynamic Threshold to trigger an alert when the calculated metric is outside of the `Upper and lower` **Decision Bounds** using a **Sensitivity** of `2` and a `slow` **Adaption Rate**.

   ![The dynamic threshold configuration with upper and lower decision bounds, sensitivity 2, and a slow adaption rate](https://mintcdn.com/validio/36lh-6_uXUsulPvT/images/docs/tutorials/tutorial-dynamic-threshold-config.png?fit=max\&auto=format\&n=36lh-6_uXUsulPvT\&q=85\&s=4406d4f96faf4a48fa8ba20aca7f381c)

   Dynamic threshold configuration
5. Under **Validator details**, update the **Priority** to Critical. This priority will be inherited by any incidents detected by this validator.
6. Click **Continue** to create the *Mean of "total sales amount"* validator.

> [!NOTE]
>
> This is just an example of one validator configuration. You can configure many validators on a single source, for different use cases such as tracking pipeline health, data consistency, completeness, and more.
>
> For more information about validator types and calculated metrics, see [Validator Types](https://docs.validio.io/v11.0/docs/validators/validator-types.md) and [Configuring a Validator](https://docs.validio.io/v11.0/docs/validators/validators-configuration.md) in our product documentation.

## Start the source

Navigate to the **Validators** tab for the source you created, for example: **Sources > gold\_\_sales\_summary > Validators**, and **Start** the source. It should take a few minutes for the backfill (historic data) to populate the graph.

On the Validators tab, select the validator you created, for example: **Mean of "total\_sales\_amount"**, to open its details page. On the **Metrics** tab, you can review the validator's calculated Data Quality over time, the daily values of the calculated metric, and the detected incidents. Once the backfill completes, you should see the daily metric plotted against the dynamic threshold's shaded bounds, with any out-of-bounds points marked as incidents.

![The Metrics tab showing the daily mean plotted against the dynamic threshold bounds with incidents marked](https://mintcdn.com/validio/36lh-6_uXUsulPvT/images/docs/tutorials/tutorial-validator-metrics.png?fit=max\&auto=format\&n=36lh-6_uXUsulPvT\&q=85\&s=6514993d0f9101c9ddcb7e34a90a3019)

Metric details for Mean of "total\_sales\_amount" validator

A vertical dashed line on the graph marks where the dynamic threshold model retrained or recalibrated — for example after a shift in the data pattern — so you can see when and why the bounds changed.

> [!NOTE]
>
> For more information, see [Validator Details Page](https://docs.validio.io/v11.0/docs/validators/validator-details.md#validator-details-page).

## Next steps

This guide showed you how to set up monitoring and validation on a data source. In the next part of the tutorial, you will triage, investigate, and troubleshoot critical incidents using Root Cause Analysis.