HomeDemoContact

BigQuery

Suggest Edits

Prerequisite credentials

You need the following to configure BigQuery credentials in the Validio platform:

  • A service account with permissions to access the specified dataset and table.
  • A base64 encoded service account key in JSON format.

📘

Permissions for service accounts

Certain permissions and roles are required for the service account.
For more information, refer to GCP - Introduction to IAM.

Service account for source config

We recommend that you create a service account with read access for the Validio platform to ingest data from your BigQuery table.

The following roles must be assigned to the service account:

  • BigQuery Data Viewer roles/bigquery.dataViewer
  • BigQuery Job User roles/bigquery.jobUser

Service account key

  • Obtain a service account key in JSON file format for your service account.
    For details, refer to Create and delete service account keys.
  • Provide the service account to the Credentials field in Validio by either:
    • Upload the JSON file to the JSON file field.
    • Paste the content of the JSON file into the JSON text field.

Credential parameters

FieldRequiredDescriptionExample
Name✅Identifier for the credentials. Used when accessing sources.service_acount_product_staging
Service account✅Upload the JSON file or paste the content of the JSON file containing the service account key.

Configuration parameters

FieldRequiredDescriptionExample
Name✅Identifier for the connector. Used when setting up validators.East_coast_weather_forecast
Project id✅Name of BigQuery project.weather-forecast
Dataset✅Name of the dataset where the table is included.east-coast
Table✅Name of the table in the dataset to ingest data from.train-data
Cron presetDetermines how often to query the bucket for new data based on a preset option.

Select custom to use your own cron expression.
Cron expression✅Determines how often to query the table for new data based on cron expression.

Expression of cron presets are displayed here.

Used to enter your own cron expression.

Miscellaneous

  • RECORD data types and REPEATABLE columns are encoded as STRING when processing.

Updated about 17 hours ago