The Amazon Kinesis integration enables real-time data quality monitoring on streaming data in Validio. To integrate with Kinesis, you will need to add a credential and add a source for the data stream.

Prerequisites for Integrating with Kinesis

Prepare credentials and permission in the AWS Management Console for Validio to validate your data:

• An IAM user with access and permissions to the specified Kinesis stream.
• Credentials for the IAM user. For more information, refer to Actions, resources, and condition keys for Amazon Kinesis Data Streams.

IAM user for Source config

We recommend that you create an IAM user where the Validio platform is granted access to the Kinesis stream you want to read data from.

The following permissions must be assigned to your IAM user:

Access key

You must supply an Access key and secret key to authenticate to the Kinesis stream. For more information, refer to Managing access keys for IAM users.

Add a Kinesis Credential

To add a Kinesis credential in Validio,

1. Navigate to Credentials and click + New Credential.
2. Under Namespace, select a namespace where the resources will be created.
3. For Credential Type, select Amazon Kinesis.
4. Fill in the Configuration parameter fields. Refer to the Kinesis Credential Parameters section.
5. (Optional) Click Test credential to validate that Validio can successfully access the Kinesis stream. If validation fails, check that you provided the correct parameter values.
6. Check Use for catalog and schema checks to automatically discover credentials and add them to the catalog page.
7. Click Create.

Kinesis Credential Parameters

Add a Kinesis Source

To add a source for Kinesis,

1. Navigate to Sources and click + New Source.
2. Under Source type, select Amazon Kinesis.
3. Fill in the configuration parameters. For more information, see Kinesis Configuration Parameters.
4. Click Continue to create the source.

Kinesis Configuration Parameters