Google Cloud Storage (GCS)
Prepare credentials and permission in Google Cloud console
Certain credentials and permission are required for Validio to validate your data:
- A service account with access and permissions to the specified GCS bucket.
- A JSON file containing the service account key.
Permissions for service accounts
Certain permissions and roles are required for the service account.
For more information, refer to GCP - Introduction to IAM.
Service account for Source config
We recommend that you create a service account where the Validio platform is granted access to the GCS bucket you want to read data from.
The following roles must be assigned to the service account:
storage.buckets.getstorage.objects.getstorage.objects.list
Service account key
- Obtain a service account key in JSON file format for your service account.
For details, refer to Create and delete service account keys. - Provide the service account to the Credentials field in Validio by either:
- Upload the JSON file to the
JSON filefield. - Paste the content of the JSON file into the
JSON textfield.
- Upload the JSON file to the
Credential parameters
| Field | Required | Description | Example |
|---|---|---|---|
| Name | ✅ | Identifier for the credentials. Used when accessing Sources. | service_account_product_staging |
| Service account | ✅ | Upload the JSON file or paste the content of the JSON file containing the service account key. |
Configuration parameters
| Field | Required | Description | Example |
|---|---|---|---|
Name |
✅ |
Identifier for the Source. Used when setting up validators. |
|
Project Id |
✅ |
Identifier of the GCS project. |
|
Bucket |
✅ |
Name of the GCS bucket that contains the folder. |
|
Folder |
✅ |
Name of the folder to read data from. |
|
File pattern |
Filter what files to read, based on file names and regex expressions. |
file_chunk_number_d+.csv |
|
Cron preset |
Determines how often to query the bucket for new data based on a preset option. Select custom to use your own cron expression. |
||
✅ |
Determines how often to query the bucket for new data based on cron expression. Expression of cron presets are displayed here. |
||
File format |
✅ |
Select the type of file: CSV, Parquet, or JSON. For CSV file formats, specify the delimiter used in the CSV file and (optional) the character or string used to represent a null value. |
CSV delimiter: , |
Updated 3 months ago