HomeDocumentationRecipesChangelog
HomeRequest DemoContact
Documentation
HomeRequest DemoContact
Start typing to search…

Azure Synapse

Create a source to monitor data from Azure Synapse.

Prerequisites for Adding an Azure Synapse Source

Azure Synapse is a Business Intelligence Tool that works with SQL Server, with two options: serverless (on-demand) SQL pool and dedicated SQL pool.

To connect to Azure Synapse, you need existing Microsoft Entra ID or Microsoft SQL Server accounts. These accounts need to have the permissions listed in the following table to access databases, schemas, and tables within your Synapse workspace. For more information, see Synapse Database Objects and Microsoft Azure documentation.

❗️

Credential Permission Requirements

Validio Credentials require VIEWER access rights when connecting to sources to read and access data. Admins must ensure that they do not provide EDITOR access rights to their credentials.


Authorization Type

Permissions Needed

Synapse with Server-less SQL Pool

(On-demand)

Synapse with Dedicated SQL Pool

Microsoft Entra ID

Synapse SQL Administrator
Synapse Administrator Storage Blob Data Reader (see Note 1)

All Validio features
(Some limitations with Lineage and Catalog, see Note 2)

All Validio features

Microsoft SQL Server

On all databases:
SELECT VIEW DATABASE STATE VIEW DEFINITION

All Validio features
(Some limitations with Lineage and Catalog, see Note 2)

All Validio features
(Without VIEW DATABASE STATE permissions, no Lineage)

📘

Notes

  1. The Storage Blob Data Reader permission is needed on the blob storage services containing external tables. Microsoft SQL Server credentials (login and password) cannot feature or read data from external tables.
  2. For authorization with Microsoft Entra ID for Synapse workspaces on a server-less pool, you will be able to view all catalog assets in Lineage (as nodes) and table-view relationships (as edges), but not table-table relationships (as edges). Also, the utilization metrics in Catalog will only show the count of reads, because writes are handled outside of the Synapse (updating the files in the blog store).

Add Azure Synapse Credentials

To monitor Azure Synapse Analytics in Validio, add valid credentials for either Microsoft Entra ID or Microsoft SQL Server. Refer to the following credential parameters tables. For more information, see About Credentials.

👍

Recommendation

For Synapse workspaces on a server-less pool, the Microsoft SQL Server account may not have access to the data. We recommend creating your credential with Microsoft Entra ID authorization.

Microsoft Entra ID Credential Parameters

FieldDescription
Credential TypeAzure Synapse Entra ID Credential
NameIdentifier for the credentials. Used when accessing sources.
Client IDApplication (client) ID which you can find in your Azure Portal.
Client SecretClient secret value for your application, which you can find in your Azure Portal.
HostDNS hostname or IP address to the database server.
PortPort number of the database server.
Database(Optional) Name of the Azure Synapse database.
Azure Synapse Backend(Optional) Specify whether the backed is a Serverless SQL Pool or Dedicated SQL Pool.

Microsoft SQL Server Credential Parameters

FieldDescription
Credential TypeAzure Synapse SQL Credential
NameIdentifier for the credentials. Used when accessing sources.
UsernameSQL Server username.
PasswordSQL Server password.
HostDNS hostname or IP address to the SQL server.
PortPort number of the SQL server.
Database(Optional) Name of the Azure Synapse database.
Azure Synapse Backend(Optional) Specify whether the backed is a Serverless SQL Pool or Dedicated SQL Pool.

Add an Azure Synapse Source

To add a source for Azure Synapse,

  1. Navigate to Observability > Sources and click + New Source.
  2. Under Source type, select Azure Synapse.
  3. Under Credentials,
    1. Select Use existing credential.
    2. Select the valid credential from the list.
  4. Under Config,
    1. Enter a Name for the source.
    2. Enter the Database, Schema, and Table to specify the data to read for the source. For more information, see the Configuration Parameters table.
    3. Select the Cron preset to specify the polling frequency for the data.
  5. Under Schema,
    1. Select the Fields to validate the new source.
    2. (Optional) Specify a timestamp field to use as the Cursor field and a Lookback time to indicate how far back to read data from the source. For more information, see Data Warehouses.
  6. Under Window,
    1. Select the Window type.
    2. Specify the Date-time field.
    3. Specify the Window size and Unit to define the period of time to validate the source.
  7. Click Continue to create the source.

Azure Synapse Configuration Parameters

Field

Description

Name

Identifier for the source. Used when setting up validators.

Database

Name of the Azure Synapse database.

Schema

Name of the schema that contains the table to access.

Table

Name of the table to read data from.

Cron preset

Defines the polling interval, or how often to query the bucket for new data.

Select Custom to use your own cron expression.

Cron expression

Automatically assigned based on the selected Cron preset. You can customize the expression.

Synapse Database Objects

Validio queries specific database objects to populate Catalog and Lineage pages after the source is added.

Catalog:

  • sys.databases
  • sys.schemas
  • sys.objects
  • sys.tables
  • sys.columns
  • sys.extended_properties
  • sys.types
  • INFORMATION_SCHEMA.TABLES

Lineage:

  • sys.dm_exec_requests_history
  • sys.dm_pdw_exec_requests
  • sys.dm_pdw_exec_sessions

Updated 1 day ago