> ## Documentation Index
> Fetch the complete documentation index at: https://docs.validio.io/llms.txt
> Use this file to discover all available pages before exploring further.

# dbt Core

> Integrate Validio with dbt Core to validate metadata for model runs and test results.

The dbt Core integration connects Validio to your local dbt projects for data transformation monitoring. Validio validates metadata for test results and model runs. To integrate with dbt Core,

1. [Add a warehouse credential](#add-a-warehouse-credential) for the warehouse where your dbt output is stored.
2. [Add a dbt Core credential](#add-a-dbt-core-credential) which will be used when uploading your dbt files.
3. [Upload your dbt manifest](#upload-dbt-files-to-validio) and run results files to Validio.
4. [Add dbt sources](#add-a-dbt-source) for monitoring model runs and test results.

Validio also supports integrating with the managed service version of dbt. For more information, see [dbt Cloud](https://docs.validio.io/v10.3/docs/integrations/data-transformation/dbt-cloud.md).

## Prerequisites for integrating with dbt Core

You must have the Validio CLI or Validio SDK installed on the machine or image that runs dbt. You can use either service to upload your dbt manifest and run results files to Validio. (Installing the CLI will automatically install the Validio SDK.) For more information, see [Getting Started with Validio Developer Tools](https://docs.validio.io/v10.3/developer/getting-started.md) and [Validio CLI](https://docs.validio.io/v10.3/developer/validio-cli.md).

## Add a warehouse credential

Before creating a dbt Core credential, you must create a valid credential for the warehouse where your dbt output is stored. You will reference this warehouse credential when you configure the dbt Core credential. For specific credential setup instructions, see [Data Warehouse Sources](https://docs.validio.io/v10.3/docs/integrations/data-warehouses.md) and select the relevant warehouse (for example, [BigQuery](https://docs.validio.io/v10.3/docs/integrations/data-warehouses/bigquery.md) or [PostgreSQL](https://docs.validio.io/v10.3/docs/integrations/transactional-databases/postgresql.md)).

## Add a dbt Core credential

To add a credential for monitoring dbt sources,

1. Navigate to **Credentials** and click **+ New credential**.
2. Under **Namespace**, select a namespace where the resources will be created.
3. For **Credential Type**, select **dbt Core**.
4. Fill in the **Configuration** parameter fields. Refer to the [dbt Core Credential Parameters](#dbt-core-credential-parameters) table.
5. (Optional) Click **Test credential** to validate the connection. If validation fails, check that you provided the correct parameter values.
6. Click **Create credential**.

Unlike with the dbt Cloud credential, Validio cannot automatically fetch the data from your dbt Core account. Instead, you must manually upload your dbt files for model runs and test results to Validio.

### dbt Core credential parameters

| Field | Description |
| :- | :- |
| Name | Identifier for the credential. This name will be referenced when creating a source. |
| Warehouse credential | Credential for the warehouse where your dbt output is stored. |

## Upload dbt files to Validio

You can use either the Validio CLI or Validio SDK to upload your dbt manifest and run files to Validio.

### Upload with Validio CLI

In the Validio CLI, run the `validio dbt upload` command as shown in the following example. For available options, refer to the dbt CLI Parameters table.

```shell title="Shell" theme={null}
DBT_CREDENTIAL_ID="my-dbtCore-credential_12345"

validio dbt upload --target-path ./target --job-name local-test --credential-id $DBT_CREDENTIAL_ID
```

| CLI Parameter | Description |
| :- | :- |
| `credential-id` | The ID or name for the dbt Core credential. |
| `job-name` | A unique name or identifier for the job. |
| `target-path` | The path to the directory containing your dbt JSON artifacts (for example, `manifest.json` and `run_results.json` files). By default, this path is called `target`. For more details, refer to [dbt documentation](https://docs.getdbt.com/reference/global-configs/json-artifacts). |

### Upload with Validio SDK

The following is an example of how to use the Validio SDK to upload the dbt files to Validio. For available options, refer to the dbt SDK Parameters table.

```python title="Python" theme={null}
import asyncio
from pathlib import Path

from validio_sdk import dbt, util
from validio_sdk.client import Client


async def main():
    client = Client()
    await dbt.upload_artifacts(
        client=client,
        credential_id="my-dbtCore-credential_12345",
        job_name="local-test",
        manifest=util.read_json_file(Path("./target/manifest.json")),
        run_results=util.read_json_file(Path("./target/run_results.json")),
    )


asyncio.run(main())
```

| dbt SDK Parameter | Description |
| :- | :- |
| `client` | A Validio `Client` instance used to authenticate the upload. |
| `credential_id` | The ID or name of the dbt Core credential. |
| `job_name` | A unique name or identifier for the job. |
| `manifest` | The parsed contents of the `manifest.json` file. |
| `run_results` | The parsed contents of the `run_results.json` file (optional). |

## Add a dbt source

Validio provides two types of dbt sources that you can add for monitoring dbt runs and tests: **dbt Model Run** and **dbt Test Result**.

To add a dbt source:

1. Navigate to **Sources** and click **+ New source**.
2. Under **Source type**, click **Data transformation** to select **dbt Model Run** or **dbt Test Result**.
3. Under **Config**,
   1. Select the valid **Credential**, or create a new credential to authenticate your connection.
   2. Select a **Project name** and **Job name**. These correspond to the data you uploaded with `validio dbt upload` (the `--job-name` you specified).
   3. Set the **Polling schedule** (cron expression in UTC), which is how frequently Validio checks for new dbt metadata.
4. Under **Source details**,
   1. Enter a **Name** for the source.
   2. (Optional) Add a **Description**.
   3. Add **Tags** to help group related sources or to use for routing notifications.
5. Click **Continue** to create the source.

You can view the summary of the source configuration.

> [!NOTE]
>
> Validio automatically infers the schema and defines default segmentation.
>
> * For dbt Model Run sources, the default segment is `model_id`.
> * For dbt Test Result sources, the default segments are `model_id` and `test_name`.