HomeDocumentationRecipesChangelog
HomeRequest DemoContact
Documentation
HomeRequest DemoContact

Prepare credentials and permission in AWS Management Console

Certain credentials and permission are required for Validio to validate your data:

  • A IAM user with access and permissions to the specified S3 bucket.
  • Credentials for the IAM user.

📘

Permissions for IAM user

Certain permissions and roles are required for the IAM user.
For more information, refer to Actions, resources, and condition keys for Amazon S3.

IAM user for Source config

We recommend that you create an IAM user where the Validio platform is granted access to the S3 bucket you want to read data from.

The following permissions must be assigned to your IAM user:

  • s3:ListBucket
  • s3:GetObject

Access key

You must supply an Access key and secret key to authenticate to the S3 bucket.
For more information, refer to Managing access keys for IAM users.

Credential parameters

FieldRequiredDescriptionExample
NameIdentifier for the credentials. Used when accessing Sources.service_acount_product_staging
Access keyAccess key with permissions to access the S3 bucket.AKIAIOSFODNN7EXAMPLE
Secret keySecret key for the specified access key.wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Configuration parameters

FieldRequiredDescriptionExample
NameIdentifier for the Source. Used when setting up Validators.
BucketName of the S3 bucket to read data from.
PrefixPrefix of the folder in the S3 bucket.If “/a/b” is specified as a prefix, Validio only reads the files that are located in the “/a/b” folder
File patternFilter what files to read, based on file names and regex expressions.file_chunk_number_\d+.csv
Cron presetDetermines how often to query the bucket for new data based on a preset option.

Select custom to use your own cron expression.
Cron expressionDetermines how often to query the bucket for new data based on cron expression.

Expression of cron presets are displayed here.

Used to enter your own cron expression.
File formatSelect the type of file: CSV, Parquet, or JSON.

For CSV file formats, specify the delimiter used in the CSV file and (optional) the character or string used to represent a null value.
CSV delimiter: ,
Null marker: NULL