Athena
Create an Athena Source
Prepare credentials and permissions in AWS Management Console
Certain credentials and permission are required for Validio to validate your data:
- An IAM user with access and permissions to query data using Athena.
- Credentials for the IAM user.
Access key
You must supply an Access key and Secret key to authenticate to AWS.
For more information, refer to Managing access keys for IAM users.
Permissions
The following permissions must be assigned to your IAM user:
Athena
athena:ListDataCatalogs
athena:ListDatabases
athena:ListTableMetadata
athena:ListQueryExecutions
athena:GetDataCatalog
athena:StartQueryExecution
athena:StopQueryExecution
athena:GetQueryExecution
athena:GetQueryResults
Glue
glue:GetTables
glue:GetTable
glue:BatchGetPartition
glue:GetDatabase
glue:GetDatabases
glue:GetPartition
glue:GetPartition
S3: Read permissions on both data source bucket and query results bucket
s3:GetBucketLocation
s3:GetObject
s3:ListBucket
s3:ListBucketMultipartUploads
s3:ListMultipartUploadParts
S3: Write permissions to query result bucket
s3:PutObject
s3:AbortMultipartUpload
For more information, refer to Identity and access management in Athena.
Credential parameters
Field | Required | Description | Example |
---|---|---|---|
Name | ✅ | Identifier for the credentials. Used when accessing Sources. | service_acount_product_staging |
Access key | ✅ | Access key for AWS authentication | AKIAIOSFODNN7EXAMPLE |
Secret key | ✅ | Secret key for the specified access key. | wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY |
AWS region | ✅ | Region from which Athena should run queries | eu-central-1 |
Query result location | ✅ | Location where to store query results | s3://myathenabucket/results |
Configuration parameters
Field | Required | Description |
---|---|---|
Name | ✅ | Identifier for the Source. Used when setting up Validators. |
Catalog | ✅ | Name of the catalog. This is sometimes called Data source . |
Database | ✅ | Name of the database. This is sometimes called schema . |
Table | ✅ | Name of the table with data to validate. |
Updated over 1 year ago