HomeDemoContact

Kinesis

Suggest Edits

Prerequisite credentials

You need the following to configure Kinesis credentials in the Validio platform:

  • A IAM user with permissions to access the specified Kinesis stream
  • Credentials for the IAM user

📘

Permissions for IAM user

Certain permissions and roles are required for the IAM user.
For more information, refer to Actions, resources, and condition keys for Amazon Kinesis.

IAM user for Source config

We recommend that you create an IAM user with access to read records in a stream, for the Validio platform to ingest data from your Kinesis stream.

The following permissions must be assigned to your IAM user:

  • Action DescribeStreamSummary; access level Read; resource type stream
  • Action ListShards; access level List
  • Action GetShardIterator; access level Read; resource type stream
  • Action GetRecords; access level Read; resource type stream

Access key

You must supply an Access key and secret key to authenticate to the S3 bucket.
For more information, refer to Managing access keys for IAM users.

Credential parameters

FieldRequiredDescriptionExample
Name✅Identifier for the credentials. Used when accessing sources.East coast weather forecast
Access key✅Access key with permissions to access the Kinesis stream.AKUEBNDJHA7BJSHJVA6F
Secret key✅Secret key for the specified access key.aslk/KOJHJKBuibhj

2jq3csaVGHYTFgvhc

ascuyg

📘

Ingestion in JSON format

Validio currently supports ingesting a Kinesis record in JSON format.

Configuration parameters

FieldRequiredDescriptionExample
Name✅Identifier for the connector. Used when setting up Validators.
AWS region✅Region that the Kinesis stream is available in.eu-north-1
Stream name✅Name of the stream to ingest data from.east-coast-weather

Inferring a schema

Based on your data, Validio helps to infer a schema of your source connector.

The automatic schema inference requires events in the stream when you connect your source. The stream can be considered empty by the Validio platform in the following circumstances:

  • A steam has no events when connecting to Validio. For example, all events are deleted according to the retention period, and no new events have been published.

Updated 9 days ago