OpenLineage Integration

Validio supports ingesting lineage events from OpenLineage-compatible tools. When your data pipelines emit OpenLineage events, Validio processes them to automatically create catalog assets, establish lineage relationships, and import dataset metadata. This enables lineage visibility for pipelines orchestrated by tools such as Apache Airflow, Apache Spark, dbt, Great Expectations, and any other tool that implements the OpenLineage standard.

Prerequisites

An API key with catalogAssets:WRITE permission. See Managing API Keys for how to create and manage API keys.

Configure your OpenLineage producer

Point your OpenLineage-compatible tool at the Validio endpoint. The configuration depends on your producer, but all use the same endpoint URL and authentication.

Endpoint

Text

POST https://your-validio-instance/openlineage/v1/lineage

Replace your-validio-instance with the host you sign in to Validio with.

Authentication

Include your API key credentials in the Authorization header. The credentials are always in the format <accessKeyId>:<secretAccessKey>. Validio supports three header formats:

Format	Header Value	Description
Raw	`<accessKeyId>:<secretAccessKey>`	Recommended. Send credentials directly as the header value.
Basic	`Basic <base64 of accessKeyId:secretAccessKey>`	Standard HTTP Basic authentication with base64-encoded credentials.
Bearer	`Bearer <accessKeyId>:<secretAccessKey>`	Bearer token format with raw credentials.

For producer-specific configuration, consult the OpenLineage integrations documentation and configure the HTTP transport with the Validio endpoint URL and API key authentication shown above.

Event requirements

Validio processes OpenLineage RunEvents only. For Validio to pick up a RunEvent and build lineage from it, the event must include these fields:

Field	Requirement
`eventType`	`COMPLETE` builds lineage. `START` and `RUNNING` only hold the run’s datasets until a `COMPLETE` event with the same `run.runId` arrives. `FAIL` and `ABORT` discard the datasets.
`eventTime`	An ISO 8601 timestamp with a time zone, for example `2026-10-08T12:00:00Z`.
`run.runId`	A UUID.
`job.namespace`, `job.name`	Any string.

The endpoint returns 200 even for events it doesn’t process, such as JobEvents, DatasetEvents, and RunEvents that are missing a required field. The same applies to datasets whose namespace or name can’t be mapped to an asset. A 200 response doesn’t confirm that lineage was created. To confirm, find the asset in Catalog and check its lineage.

How datasets map to catalog assets

Every OpenLineage dataset has a namespace and a name. Validio uses these two values to decide which catalog asset the dataset belongs to. Off-the-shelf OpenLineage integrations already follow the OpenLineage naming conventions. If you write your own producer, use the formats below so your datasets map to the correct assets.

Namespace

For data warehouses, Validio reads only the scheme of the namespace, which is the part before ://. The host, port, and path are ignored, so you can’t specify the database in the namespace. Validio recognizes the scheme regardless of case.

Scheme example	Maps to
`awsathena://`	Amazon Athena
`azure://`	Azure Synapse
`bigquery`	Google BigQuery
`clickhouse://`	ClickHouse
`databricks://`	Databricks
`db2://`	IBM Db2
`openedge://`	Progress OpenEdge
`oracle://`	Oracle
`postgres://`	PostgreSQL
`redshift://`	Amazon Redshift
`sapiq://`	SAP IQ
`snowflake://`	Snowflake
`mssql://`	Microsoft SQL Server
`teradata://`	Teradata

Namespaces with any other scheme become generic assets. See Other Schemes.

Name

For data warehouses, the dataset name must be database.schema.table. Teradata also accepts database.table. Validio skips datasets whose names have fewer parts and creates no asset or lineage for them.

Namespace	Name	Result
`postgres://db.example.com:5432`	`analytics.public.orders`	Mapped to the `analytics.public.orders` table.
`postgres://db.example.com:5432/analytics`	`public.orders`	Skipped. The database in the namespace path is ignored, and the name has only two parts.

Matching existing assets

Validio matches a warehouse dataset to a catalog asset by the warehouse type and the database.schema.table name, regardless of case. If a matching table is already in the catalog, the lineage attaches to that asset. If not, Validio creates a new asset, and the next catalog refresh fills in its details.

Other schemes

Datasets with a scheme that isn’t a supported data warehouse become generic assets:

Object storage (s3, gcs, gs, abfs, abfss, wasb, wasbs, dbfs): the asset is named <namespace>/<name>, for example s3://my-bucket/raw/orders.
Any other scheme: the asset is named <scheme>://<name>, for example kafka://orders-topic. The host and path of the namespace are left out of the name.

Columns

Validio matches column names from the schema and column lineage facets to the asset’s existing columns. If Validio doesn’t recognize a column name, it adds it as a new column, so a typo creates an extra column instead of an error. Use the column names exactly as they appear in Catalog, including case, so that column descriptions appear on the correct columns.

Example event

The following COMPLETE event creates a table-level lineage edge from RAW.SALES.ORDERS to ANALYTICS.SALES.DAILY_ORDERS in Snowflake. It also creates a column-level edge from ORDER_ID to ORDER_ID.

JSON

{
  "eventType": "COMPLETE",
  "eventTime": "2026-10-08T12:00:00Z",
  "run": { "runId": "3f1e9c62-8a4b-4d2e-9f7a-1b2c3d4e5f60" },
  "job": { "namespace": "my-pipeline", "name": "load_daily_orders" },
  "producer": "https://example.com/my-producer",
  "schemaURL": "https://openlineage.io/spec/2-0-2/OpenLineage.json#/$defs/RunEvent",
  "inputs": [
    { "namespace": "snowflake://myorg-myaccount", "name": "RAW.SALES.ORDERS" }
  ],
  "outputs": [
    {
      "namespace": "snowflake://myorg-myaccount",
      "name": "ANALYTICS.SALES.DAILY_ORDERS",
      "facets": {
        "columnLineage": {
          "_producer": "https://example.com/my-producer",
          "_schemaURL": "https://openlineage.io/spec/facets/1-2-0/ColumnLineageDatasetFacet.json",
          "fields": {
            "ORDER_ID": {
              "inputFields": [
                {
                  "namespace": "snowflake://myorg-myaccount",
                  "name": "RAW.SALES.ORDERS",
                  "field": "ORDER_ID"
                }
              ]
            }
          }
        }
      }
    }
  ]
}

What Validio processes

When Validio receives an OpenLineage event, it extracts:

Lineage relationships — Input and output datasets from the event are mapped to catalog assets, and lineage edges are created between them.
Dataset metadata — Descriptions, tags, and other metadata from OpenLineage facets are imported and associated with the corresponding catalog assets.
Run state — The run identifier and event type (START, RUNNING, COMPLETE, FAIL, ABORT) are used only to collect a run’s datasets until the run finishes. Validio builds lineage when it receives the COMPLETE event, then discards the run state. A run that ends with FAIL or ABORT adds no lineage. Run status and duration are not stored or shown in Validio.

How it appears in Validio

Catalog assets

Assets discovered through OpenLineage events appear in your Catalog alongside assets from other credential types. Warehouse datasets appear as tables under their database and schema. Other datasets appear as generic assets. See How Datasets Map to Catalog Assets.

Lineage

Lineage edges created from OpenLineage input/output relationships are visible in the Lineage graph. These edges behave like any other lineage edge — you can inspect them, add descriptions, and use them for glossary term propagation.

Descriptions

Descriptions imported from OpenLineage metadata appear with the OpenLineage origin label, alongside descriptions from other sources such as dbt and Catalog Refresh. OpenLineage descriptions are read-only. To customize a description, edit it and a Validio copy is created that takes priority in the display. For more information about description origins, see Catalog Assets.

Introduction

Using Validio

Validio Tutorials

AI Features

Credentials

Sources

Validators

Catalog

Lineage

Incidents & Errors

Notifications

Integrations

Schema & Profiling

Filters

Windows

Segmentation

Thresholds

Access & Settings

Deployment

Security

Other

OpenLineage

Prerequisites

Configure your OpenLineage producer

Endpoint

Authentication

Event requirements

How datasets map to catalog assets

Namespace

Name

Matching existing assets

Other schemes

Columns

Example event

What Validio processes

How it appears in Validio

Catalog assets

Lineage

Descriptions

​Prerequisites

​Configure your OpenLineage producer

​Endpoint

​Authentication

​Event requirements

​How datasets map to catalog assets

​Namespace

​Name

​Matching existing assets

​Other schemes

​Columns

​Example event

​What Validio processes

​How it appears in Validio

​Catalog assets

​Lineage

​Descriptions

​Related resources

Prerequisites

Configure your OpenLineage producer

Endpoint

Authentication

Event requirements

How datasets map to catalog assets

Namespace

Name

Matching existing assets

Other schemes

Columns

Example event

What Validio processes

How it appears in Validio

Catalog assets

Lineage

Descriptions

Related resources