HomeDemoContact

GCS

Suggest Edits

Prerequisite credentials

You need the following to configure GCS credentials in the Validio platform:

  • A service account with permissions to access the specified GCS bucket
  • A base64 encoded service account key in JSON format

📘

Permissions for service accounts

Certain permissions and roles are required for the service account.
For more information, refer to GCP - Introduction to IAM.

Service account for Source config

We recommended that you create a service account with read access for the Validio platform to ingest data from your GCS bucket.

The following roles must be assigned to the service account:

  • storage.buckets.get
  • storage.objects.get
  • storage.objects.list

Service account key

  • Obtain a service account key in JSON file format for your service account.
    For details, refer to Create and delete service account keys.
  • Provide the service account to the Credentials field in Validio by either:
    • Upload the JSON file to the JSON file field.
    • Paste the content of the JSON file into the JSON text field.

Credential parameters

FieldRequiredDescriptionExample
Name✅Identifier for the credentials. Used when accessing sources.service_acount_product_staging
Service account✅Upload the JSON file or paste the content of the JSON file containing the service account key.

Configuration parameters

FieldRequiredDescriptionExample
Name✅Identifier for the connector. Used when setting up validators.
Project Id✅Identifier of the project containing the GCS bucket to ingest data from.
Bucket✅Name of the GCS bucket to ingest data from.
Folder✅Name of the folder in the bucket to ingest data from.
File patternFilter what files to ingest, based on file names and regex expressions.file_chunk_number_\d+.csv
Cron presetDetermines how often to query the bucket for new data based on a preset option.

Select custom to use your own cron expression.
Cron expression✅Determines how often to query the bucket for new data based on cron expression.

Expression of cron presets are displayed here.
Used to enter your own cron expression.
CSV parser config✅Specify which delimiter is used in the CSV file.

Optionally, specify a character or string used to represent Null.		CSV delimiter: ,

Null marker: NULL

Updated about 16 hours ago