BigQuery
Prerequisite credentials
You need the following to configure BigQuery credentials in the Validio platform:
- A service account with permissions to access the specified dataset and table.
- A base64 encoded service account key in JSON format.
Permissions for service accounts
Certain permissions and roles are required for the service account.
For more information, refer to GCP - Introduction to IAM.
Service account for source config
We recommend that you create a service account with read access for the Validio platform to ingest data from your BigQuery table.
The following roles must be assigned to the service account:
- BigQuery Data Viewer
roles/bigquery.dataViewer
- BigQuery Job User
roles/bigquery.jobUser
Service account key
- Obtain a service account key in JSON file format for your service account.
For details, refer to Create and delete service account keys. - Provide the service account to the Credentials field in Validio by either:
- Upload the JSON file to the
JSON file
field. - Paste the content of the JSON file into the
JSON text
field.
- Upload the JSON file to the
Credential parameters
Field | Required | Description | Example |
---|---|---|---|
Name | β | Identifier for the credentials. Used when accessing sources. | service_acount_product_staging |
Service account | β | Upload the JSON file or paste the content of the JSON file containing the service account key. |
Configuration parameters
Field | Required | Description | Example |
---|---|---|---|
Name | β | Identifier for the connector. Used when setting up validators. | East_coast_weather_forecast |
Project id | β | Name of BigQuery project. | weather-forecast |
Dataset | β | Name of the dataset where the table is included. | east-coast |
Table | β | Name of the table in the dataset to ingest data from. | train-data |
Cron preset | Determines how often to query the bucket for new data based on a preset option. Select custom to use your own cron expression. | ||
Cron expression | β | Determines how often to query the table for new data based on cron expression. Expression of cron presets are displayed here. Used to enter your own cron expression. |
Miscellaneous
RECORD
data types andREPEATABLE
columns are encoded asSTRING
when processing.
Updated about 17 hours ago