Atlan (beta)
View Validio data quality metrics in Atlan.
Beta Feature
Beta features represent newly added features in the product, and there is a chance of incompatible changes that impact the API and functionality in subsequent releases as we approach a stable version. As such we will not provide any advanced notice of any deprecation of functionality related to beta features.
Atlan stores tables, views, columns, and lineage as assets in its data catalog. The Validio integration sends data quality metrics at regular intervals to Atlan, where you can view the metrics alongside each asset.
Example of Validio metric information viewed in Atlan
Validio sends the following aggregated metrics to Atlan:
Metric Information | Description |
|---|---|
Validio Asset URL | URL link from Atlan to the asset details page in Validio. This is provided for you to follow up or investigate the metrics shared with Atlan. |
Freshness validation | Does the asset have at least one freshness validator set up? This check includes both data and metadata-based validators. |
Row Count validation | Does the asset have at least one row count validator set up? This check includes both data and metadata-based validators. |
Custom validation | Does the asset have validations other than freshness and row count set up? |
Total metrics | The total number of validations. |
Total incidents | The total number of incidents generated on the last validation. |
Data quality score | The percentage of the ratio of validations to incident-generating validations: |
Last Updated | The date and time when the metrics were last polled and updated. |
Prerequisites for Integrating with Atlan
To connect Validio to your Atlan instance, you will need:
- The URL to your Atlan instance and an Atlan API token to create the credential in Validio.
Add a Credential for Atlan
To add an Atlan credential in Validio,
- Navigate to Credentials, and click + New Credential.
- Under Namespace, select a namespace to define where resources will be created and who can access it.
- Under Credential Type, select Atlan Credential.
- Fill in the configuration parameters for the credential. For more information, see Atlan Credential Parameters.
- Click Create.
Atlan Credential Parameters
| Parameter | Description |
|---|---|
| Name | The display name of your credential. |
| Base URL | The URL to your Atlan instance. For example: https://my-company.atlan.com |
| API Token | Your Atlan API token. If needed, you can generate a new token in Atlan, under your account Admin > Workspace > API tokens. |
| Application Link URL | The URL for your Validio application instance. For example, https://my-company.validio.io |
Asset Mapping between Validio and Atlan
The following describes how assets are mapped between Validio and Atlan:
- Atlan will discover assets for each connection or credential. If you add identical connections twice, Atlan will discover two identical assets. Validio will deduplicate assets. If two Validio credentials discover the same asset, Validio treats them as one asset.
- When mapping assets to Atlan, Validio accounts for the type, catalog, schema, and table--so, there cannot be duplicates between different warehouses, such as Snowflake and Postgres. If one Validio asset resolves to multiple Atlan assets, all Atlan assets will be updated based on the assumption that the assets are the same but originate from different workflows or connections.
Because you can add multiple sources to one asset in Validio, a single source is not always a single asset. Any aggregation on the asset-level (such as "Number of incidents" or "Has freshness") means that the incident can come from any amount of sources, and it only requires at least one of the sources to have a freshness validator, not all of them.
Updated 7 days ago