dbt Cloud
Validio supports integrating with dbt (data build tool) to validate metadata for test results and model runs. To integrate with dbt Cloud,
- Add a warehouse credential for the warehouse where your dbt output is stored.
- Add a dbt Cloud credential which will automatically fetch your data from your dbt Cloud account.
- Add dbt sources for monitoring model runs and test results.
Prerequisites for Integrating with dbt Cloud
Integrating with dbt Cloud requires a dbt Cloud API token and account ID. The token should have read-only access. For more information, see dbt Cloud’s API documentation.
Add a Warehouse Credential
Before creating a dbt Cloud credential, you must create a valid credential for the warehouse where your dbt output is stored. You will reference this warehouse credential when you configure the dbt Cloud credential. For specific credential setup instructions, see Data Warehouse Sources and select the relevant warehouse (for example, BigQuery or PostgreSQL).
Add a dbt Cloud Credential
To add a credential for monitoring dbt sources,
- Navigate to Observability > Credentials and click + Add Credential.
- Under Credential type, select dbt Cloud Credential.
- Fill in the configuration parameters. For more information, see dbt Cloud Credential Parameters.
- Click Create.
Validio will validate that the dbt Cloud account can be successfully accessed. If validation passes, Validio will automatically fetch the metadata for the runs and tests. If validation fails, check that you provided the correct parameter values and try again.
Note
Validio fetches and processes results from dbt runs and tests on a regular schedule. Depending on the schedule, it might take up to 60 minutes after the dbt Cloud credential is successfully added before you see dbt metadata.
Once the credential is created, you can add sources to track dbt Model Runs and dbt Test Results.
dbt Cloud Credential Parameters
Parameter | Description |
---|---|
Credential Type | dbt Cloud Credential |
Name | Identifier for the credential. This name will be referenced when creating a source. |
Warehouse Credential | Credential for the table where your dbt output is stored. |
Account ID | dbt Cloud Account ID. |
AI Base URL | dbt Cloud base URL. |
dbt Cloud Token | dbt Cloud API token. |
Add a dbt Source
Validio provides two types of dbt sources that you can add for monitoring dbt runs and tests: dbt Model Runs and dbt Test Results.
To add a dbt source:
- Navigate to Observability > Sources and click + New Source.
- Under Source type, select either dbt Model Runs or dbt Test Results.
- Under Credentials,
- Select Use existing Credential.
- Select the valid credential from the list.
- Under Config,
- Enter a Name for the source.
- Select a Project Name and Job Name.
- Select a Cron preset to specify the polling frequency for the data.
- Under Window, select the Window type, which should be either Tumbling Window or Global Window. For more information, see Configuring Windows.
- Click Continue to create the source.
You can view the summary of the source configuration.
Note
Validio automatically infers the schema and defines default segmentation.
- For dbt Model Runs, the default segment is
model_id
.- For dbt Test Results, the default segments are
model_id
andtest_name
.
Updated 6 months ago