HomeRequest DemoContact

Google Cloud Storage (GCS)

Suggest Edits

Prepare credentials and permission in Google Cloud console

Certain credentials and permission are required for Validio to validate your data:

  • A service account with access and permissions to the specified GCS bucket.
  • A JSON file containing the service account key.

📘

Permissions for service accounts

Certain permissions and roles are required for the service account.
For more information, refer to GCP - Introduction to IAM.

Service account for Source config

We recommend that you create a service account where the Validio platform is granted access to the GCS bucket you want to read data from.

The following roles must be assigned to the service account:

  • storage.buckets.get
  • storage.objects.get
  • storage.objects.list

Service account key

  • Obtain a service account key in JSON file format for your service account.
    For details, refer to Create and delete service account keys.
  • Provide the service account to the Credentials field in Validio by either:
    • Upload the JSON file to the JSON file field.
    • Paste the content of the JSON file into the JSON text field.

Credential parameters

FieldRequiredDescriptionExample
Name✅Identifier for the credentials. Used when accessing Sources.service_account_product_staging
Service account✅Upload the JSON file or paste the content of the JSON file containing the service account key.

Configuration parameters

FieldRequiredDescriptionExample
Name✅Identifier for the Source. Used when setting up validators.
Project Id✅Identifier of the GCS project.
Bucket✅Name of the GCS bucket that contains the folder.
Folder✅Name of the folder to read data from.
File patternFilter what files to read, based on file names and regex expressions.file_chunk_number_\d+.csv
Cron presetDetermines how often to query the bucket for new data based on a preset option.

Select custom to use your own cron expression.
Cron expression✅Determines how often to query the bucket for new data based on cron expression.

Expression of cron presets are displayed here.
Used to enter your own cron expression.
File format✅Select the type of file: CSV, Parquet, or JSON.

For CSV file formats, specify the delimiter used in the CSV file and (optional) the character or string used to represent a null value.		CSV delimiter: ,
Null marker: NULL

Updated 2 months ago