> ## Documentation Index
> Fetch the complete documentation index at: https://docs.validio.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Kafka

> Connect Validio to Apache Kafka for real-time data quality monitoring on streaming data.

The Apache Kafka integration enables real-time data quality monitoring on streaming data in Validio. To integrate with Kafka, you will need to add a credential and add a source for the data stream.

## Prerequisites for integrating with Kafka

Certain credentials and permission are required for Validio to validate your data:

* A service account with access and permissions to the specific Kafka clusters. For more information, refer to [Kafka security documentation](https://kafka.apache.org/documentation/#security) and [Confluent: Kafka SASL/PLAIN authentication](https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_plain.html).

## Add a Kafka credential

To add a Kafka credential in Validio,

1. Navigate to **Credentials** and click **+ New credential**.
2. Under **Namespace**, select a namespace where the resources will be created.
3. For **Credential Type**, select either **Kafka SSL** or **Kafka SASL SSL Plain**, depending on the authentication method for your Kafka clusters.
4. Fill in the **Configuration** parameter fields. Refer to the [Kafka Credential Parameters](#kafka-credential-parameters) section.
5. (Optional) Click **Test credential** to validate that Validio can successfully access the Kafka cluster. If validation fails, check that you provided the correct parameter values.
6. (Optional) Check **Use for catalog and schema checks** to list the cluster's topics as [catalog assets](https://docs.validio.io/v11.0/docs/catalog/catalog-assets.md). Kafka catalog assets include only the topic names, not their schemas, so Validio does not automatically detect schema changes on Kafka topics. Validio reads a topic's schema when you create a source for it.
7. Click **Create**.

### Kafka credential parameters

The Kafka credential configuration depends on the authentication method for your Kafka clusters, either `Kafka SSL` or `Kafka SASL SSL Plain`.

#### Kafka SSL parameters

The `Kafka SSL Credentials` credential type requires the following parameters and certificates:

| Field | Description |
| :- | :- |
| Name | Identifier for the credentials. Used when accessing Sources. |
| Bootstrap server | The address of Kafka load balancer or Kafka cluster address. Use a comma to separate multiple addresses. |
| CA certificate | A certificate of Certificate authority (CA) in CRT format. |
| Client certificate | Client SSL certificate in PEM format. |
| Client private key | Private keys of client certificate in PEM format. |
| Client private key password | Password or passphrase of private keys for the client certificate. |

> [!NOTE]
>
> You can either provide the certificates and client private key as plain text or upload the files.

#### Kafka SASL SSL Plain parameters

The `Kafka SASL Plain Credential` credential type, which uses `TLS/SSL` as transport layer, requires the following parameters and certificates:

| Field | Description |
| :- | :- |
| Name | Identifier for the credentials. Used when accessing Sources. |
| Bootstrap server | The address of Kafka load balancer or Kafka cluster address. Use a comma to separate multiple addresses. |
| Username | Username for the SASL Plain Credential. |
| Password | Password for the SASL Plain Credential. |

## Add a Kafka source

To add a source for Kafka,

1. Navigate to **Sources** and click **+ New source**.
2. Under **Source type**, select **Apache Kafka**.
3. Fill in the configuration parameters. For more information, see [Kafka Configuration Parameters](#kafka-configuration-parameters).
4. Click **Continue** to create the source.

### Kafka configuration parameters

| Field | Description |
| :- | :- |
| Name | Identifier for the Source. Used when setting up Validators. |
| Cluster | Name of the Kafka cluster to read data from. |
| Message format | JSON, AVRO, or PROTOBUF |
| Message schema | For JSON, Validio automatically infers the schema. For AVRO and PROTOBUF, you can upload schemas. For more information, see [Data Stream Sources](https://docs.validio.io/v11.0/docs/integrations/data-streams.md). |