Using Catalog

Data Catalog automatically discovers assets based on the Credentials you have added to Validio and enabled for catalog. The Catalog page organizes your data assets into tabs based on what they are: tables, streams, object storage services, business intelligence tools, and activities.

📘

Catalog Asset Discovery

It may take longer for the catalog assets to populate the first time that a catalog job runs for a new credential. For more information, see About Credentials.

From the Tables, Streams, Object Storage and BI Tools tabs, you can do the following:

• Search and filter to quickly find relevant assets
• View and update information about an asset
• Convert assets to a source to track and validate data from that asset
• View relationships with other assets in Lineage

Search and Filter Assets

If you have a large number of catalog assets, you can search and filter to show only relevant assets in the list. For example, you can search for specific assets using keywords and then apply quick filters to narrow your results further by selected values. You can search and filter in all the catalog assets tabs except the Activities tab.

• Catalog Search–Type keywords into the search bar and the asset list will update automatically to show matching results, highlighting the keywords that match in the Name and Description fields. When you type more than one keyword, the results will return all assets that include any of the keywords.
• Quick Filters–Selecting one or more values of Type, Credential, or Label to quickly filter the list of assets. Clicking on the filters also shows a count of the assets that match each value.

📘

Note

The results of searching and filtering will respect the sorting that has been applied to the table.

Refresh Catalog Assets

You can manually refresh your catalog to discover new assets since the last refresh. When refreshing your catalog assets, you can select specific credentials or refresh all assets.

• You can only choose credentials that have been enabled for catalog.
• A full refresh of assets performs over the past 30 days.

View Asset Details

You can view more information about the asset or source by clicking on the row in the table to open the details panel or clicking the name of the asset to open its Source details page.

• The details panel includes descriptions and metadata that is indexed and searchable. The panel also displays utilization statistics as well as configuration and ownership information. From this panel, you can also add tags, convert the asset to a source, or view the asset or source in the Lineage page.
• If you click on the asset name to open its Source details page, you will only have access to the Overview and Schema tags. You can also convert the asset to a source from this page. For more information, see About Source Details.

Add Custom Catalog Descriptions

Validio imports descriptions and tags automatically from external systems, such as BigQuery and dbt. You can also add your own descriptions to a catalog asset. All existing descriptions are included in the Description column of the assets table, with icons to indicate where they originate.

To add a custom catalog description,

1. Click the asset Name in the table to open its source details page.
2. In the Overview tab, click the edit icon next to Add description.
3. Type in your description.
4. Click the check mark to save your description.

When you return to the Catalog page, the custom description will display in the table with the Validio icon next to it.

Add Asset Tags

You can add tags organize and prioritize your data assets with descriptive labels or other relevant information. Use the details panel to add tags to an asset or source.

Convert Assets to Sources

Validio does not actively monitor or validate the data and schema of the catalog assets until you convert the asset to a source and create validators on that source. You can select different types of assets to convert to sources in one batch operation. For more information, see About Sources.

To convert a catalog asset to a source,

1. Navigate to Catalog > Tables and check rows in the table for each asset you want to convert to a source. You can convert more than one asset at a time, and the assets do not need to be the same type.
2. Click Convert to Source to open the configuration wizard.
3. Under Catalog asset,

   1. Select the credentials to associate with each catalog asset when converting to a source. The credential will authenticate your connection to the data source.

      For each asset, the credential list is filtered for the type of asset.

   2. Set how many days of historic data to use when you start the source.

   3. Specify how frequently the validators on the source will check for changes.

   4. Click Continue.

      If one or more of the assets you are converting is a BigQuery or Snowflake table, the next step will be adding recommended validators on those sources. Otherwise, this is the last step of the configuration.

4. (Optional) Under Validators, select the metadata validators to add to your BigQuery or Snowflake sources. These validators will not be added to other types of sources.
5. Click Continue to convert the asset to a source.

View Asset in Lineage

From the details panel for an asset, click View Lineage to open the Lineage page where you can view the relationships with other assets. For more information, see About Lineage.