ClickHouse

​

Prerequisites for integrating with ClickHouse

• The user profile has readonly permissions, which means that the user can only execute read queries (such as SELECT).
• The role is also granted remote permissions to fetch the query logs needed to build the lineage relationships.

CREATE ROLE IF NOT EXISTS validio_role;

CREATE USER IF NOT EXISTS validio_user
    IDENTIFIED WITH sha256_password BY '**************'
    SETTINGS PROFILE 'readonly'
    DEFAULT ROLE validio_role;

GRANT SELECT, SHOW ON mydatabase.* TO validio_role;
GRANT SELECT ON system.query_log TO validio_role;
GRANT SELECT ON system.clusters TO validio_role;
GRANT REMOTE ON *.* to validio_role;

​

Add a ClickHouse credential

1. Navigate to Credentials and click + New credential.
2. Under Namespace, select a namespace where the resources will be created.
3. For Credential Type, select ClickHouse Credential.
4. Fill in the Configuration parameter fields. Refer to the ClickHouse Credential Parameters table.
5. (Optional) Click Test credential to validate that Validio can successfully access the ClickHouse account. If validation fails, check that you provided the correct parameter values.
6. (Optional) Check Use for catalog and schema checks to automatically discover assets from this credential and add them to the Catalog page.
7. Click Create credential.

​

ClickHouse credential parameters

​

Add a ClickHouse source

1. Navigate to Sources and click + New source.
2. Under Source type, select ClickHouse.
3. Under Config,
   1. Select the valid Credential or create a new credential to authenticate your connection to the data warehouse.
   2. You have two options for specifying the dataset and tables to monitor:
      1. Select an existing table — Enter the Database, Schema, and Table to specify where the data comes from. Selecting more than one table will create a new source for each table.
      2. Use Custom SQL — Write a valid SQL query to specify the tables to monitor.
   3. Set the Polling schedule, which is how frequently the validators on the source will check for changes.
4. Under Schema, click Continue to automatically infer the schema fields from the tables you selected. If you select many tables, this operation can take a few minutes to complete.
5. Under Source details,
   1. Add Tags to help group related sources or to use for routing notifications.
   2. Add an Owner who will be the contact for incident notifications.
   3. Assign a Priority, which indicates the importance of incidents detected on this source.
6. Click Continue to create the source. Source names are generated automatically and will be displayed when the source creation completes. If there are more than 5 sources, you will see the names for the first five and a count of the remaining sources.