Prerequisite credentials

You need the following to configure BigQuery credentials in the Validio platform:

  • A service account with permissions to access the specified dataset and table.
  • A base64 encoded service account key in JSON format.

πŸ“˜

Permissions for service accounts

Certain permissions and roles are required for the service account.
For more information, refer to GCP - Introduction to IAM.

Service account for source config

We recommend that you create a service account with read access for the Validio platform to ingest data from your BigQuery table.

The following roles must be assigned to the service account:

  • BigQuery Data Viewer roles/bigquery.dataViewer
  • BigQuery Job User roles/bigquery.jobUser

Service account key

  • Obtain a service account key in JSON file format for your service account.
    For details, refer to Create and delete service account keys.
  • Provide the service account to the Credentials field in Validio by either:
    • Upload the JSON file to the JSON file field.
    • Paste the content of the JSON file into the JSON text field.

Credential parameters

FieldRequiredDescriptionExample
Nameβœ…Identifier for the credentials. Used when accessing sources.service_acount_product_staging
Service accountβœ…Upload the JSON file or paste the content of the JSON file containing the service account key.

Configuration parameters

FieldRequiredDescriptionExample
Nameβœ…Identifier for the connector. Used when setting up validators.East_coast_weather_forecast
Project idβœ…Name of BigQuery project.weather-forecast
Datasetβœ…Name of the dataset where the table is included.east-coast
Tableβœ…Name of the table in the dataset to ingest data from.train-data
Cron presetDetermines how often to query the bucket for new data based on a preset option.

Select custom to use your own cron expression.
Cron expressionβœ…Determines how often to query the table for new data based on cron expression.

Expression of cron presets are displayed here.

Used to enter your own cron expression.

Miscellaneous

  • RECORD data types and REPEATABLE columns are encoded as STRING when processing.