Kinesis
Prepare credentials and permission in AWS Management Console
Certain credentials and permission are required for Validio to validate your data:
- A IAM user with access and permissions to the specified Kinesis stream.
- Credentials for the IAM user.
Permissions for IAM user
Certain permissions and roles are required for the IAM user.
For more information, refer to Actions, resources, and condition keys for Amazon Kinesis Data Streams.
IAM user for Source config
We recommend that you create an IAM user where the Validio platform is granted access to the Kinesis stream you want to read data from.
The following permissions must be assigned to your IAM user:
| Actions | Access level | Required resource type |
|---|---|---|
SubscribeToShard | Read | consumer |
ListShards | List | stream |
DescribeStreamConsumer | Read | consumer |
ListStreamConsumers | Read | stream |
RegisterStreamConsumer | Write | stream |
ListStreams | List | |
EnableEnhancedMonitoring | Write | |
DescribeLimits | Read | |
DisableEnhancedMonitoing | Write |
Access key
You must supply an Access key and secret key to authenticate to the Kinesis stream.
For more information, refer to Managing access keys for IAM users.
Credential parameters
| Field | Required | Description | Example |
|---|---|---|---|
| Name | ✅ | Identifier for the credentials. Used when accessing Sources. | East coast weather forecast |
| Access key | ✅ | Access key with permissions to access the Kinesis stream. | AKUEBNDJHA7BJSHJVA6F |
| Secret key | ✅ | Secret key for the specified access key. | aslk/KOJHJKBuibhj 2jq3csaVGHYTFgvhc ascuyg |
Configuration parameters
| Field | Required | Description | Example |
|---|---|---|---|
| Name | ✅ | Identifier for the Source. Used when setting up Validators. | |
| AWS region | ✅ | The AWS region that the Kinesis stream is available in. | eu-north-1 |
| Stream name | ✅ | Name of the stream to read data from. | east-coast-weather |
| Message format | ✅ | JSON, AVRO or PROTOBUF | JSON |
| Message schema | ✅ | JSON - Not needed, Validio automatically infers schema AVRO - Refer to Avro Schema Declaration. PROTOBUF - Refer to Protobuf Message Type. |
Updated about 1 year ago