Prerequisite credentials
You need the following to configure GCS credentials in the Validio platform:
- A service account with permissions to access the specified GCS bucket
- A base64 encoded service account key in JSON format
Permissions for service accounts
Certain permissions and roles are required for the service account.
For more information, refer to GCP - Introduction to IAM.
Service account for Source config
We recommended that you create a service account with read access for the Validio platform to ingest data from your GCS bucket.
The following roles must be assigned to the service account:
storage.buckets.get
storage.objects.get
storage.objects.list
Service account key
- Obtain a service account key in JSON file format for your service account.
For details, refer to Create and delete service account keys. - Provide the service account to the Credentials field in Validio by either:
- Upload the JSON file to the
JSON file
field. - Paste the content of the JSON file into the
JSON text
field.
- Upload the JSON file to the
Credential parameters
Field | Required | Description | Example |
---|---|---|---|
Name | β | Identifier for the credentials. Used when accessing sources. | service_acount_product_staging |
Service account | β | Upload the JSON file or paste the content of the JSON file containing the service account key. |
Configuration parameters
Field | Required | Description | Example |
---|---|---|---|
Name | β | Identifier for the connector. Used when setting up validators. | |
Project Id | β | Identifier of the project containing the GCS bucket to ingest data from. | |
Bucket | β | Name of the GCS bucket to ingest data from. | |
Folder | β | Name of the folder in the bucket to ingest data from. | |
File pattern | Filter what files to ingest, based on file names and regex expressions. | file_chunk_number_\d+.csv | |
Cron preset | Determines how often to query the bucket for new data based on a preset option. Select custom to use your own cron expression. | ||
Cron expression | β | Determines how often to query the bucket for new data based on cron expression. Expression of cron presets are displayed here. Used to enter your own cron expression. | |
CSV parser config | β | Specify which delimiter is used in the CSV file. Optionally, specify a character or string used to represent Null. | CSV delimiter: , Null marker: NULL |
Updated about 16 hours ago