This guide walks through each of the four steps in the core Validio Code workflow:

1. Initialize–Create a Validio Code project.
2. Author–Write or modify your infrastructure as code to describe the desired resources in your environment.
3. Plan–Preview the changes made to your environment before applying.
4. Apply–When you are satisfied with the program, deploy the changes to your environment.

This guide provides descriptions and examples for different scenarios, which you can see in Validio Code Scenarios.

Initialize

To create a Validio Code project to manage your resources, you can start by scaffolding a project. In this case, a project is the directory housing the Python program responsible for managing your resources.

You can use the init command to create a new project. There are no special considerations for the initialized directory. It contains the following files: main.py and requirements.txt.

In the following example, you create a project in a directory called my-project with a namespace called my-namespace.

validio code init --directory my-project --namespace my-namespace

📘

Note

Because it is a Python project, you can introduce any Python tools, models, dependencies, IDE, and so on.

Directory

If you are creating the project from within the desired directory, you can omit the --directory flag in the init command.

Namespace

You should always assign a unique namespace to every project that you create.

To assign a namespace, add the -n or --namespace flag to the init command. For more information, see Validio API.

Author

When writing your infrastructure as code, you can use definitions provided by the Validio SDK to describe the resource types in your environment. For more information, see Validio SDK.

You can use any Python-based tools at your disposal when you author a program. For example, you can reuse models, import external dependencies, and load credentials from your preferred secrets manager.

To deploy the resource in your environment, create an instance of the class definition for the resource type you want to declare. For easier maintenance, you can divide your resource declarations and helper functionality across different modules (as you would in any program).

To delete individual resource definitions from your program, either remove or comment out the related code.

📘

Note

Environment variables passed to the plan and apply command are visible to the Validio Code program.

This example shows the contents of a working main.py file. The file consists of a full program that declares which resources to deploy into an environment:

from validio_sdk.resource.credentials import GcpCredential
from validio_sdk.resource.sources import GcpBigQuerySource
from validio_sdk.resource.segmentations import Segmentation
from validio_sdk.resource.windows import TumblingWindow, WindowTimeUnit
from validio_sdk.resource.validators import NumericMetric, NumericValidator

# Declare a GCP Credential
gcp_credential = GcpCredential(name='example-credential', credential='<svc-acct>')

# Declare a BigQuery Source
big_query_source = GcpBigQuerySource(
    name='example-source',
    credential=gcp_credential,
    project='example-project',
    dataset='example-dataset',
    table='orders',
    cursor_field='created_at',
    lookback_days=30,
    schedule='0 */12 * * *',
)

# Attach a 1 hour Tumbling window to the source
window = TumblingWindow(
    name='example-window', source=big_query_source,
    data_time_field='event_time',
    window_size=1, time_unit=WindowTimeUnit.HOUR)

# Attach a segmentation to the source
by_gender = Segmentation(name='example-segmentation', source=big_query_source, fields=['Gender'])

# Set up a couple validators on the source
for field in ['Age', 'Credit_scoring']:
    NumericValidator(
    	name=f'mean_of_{field}',
    	window=window, segmentation=by_gender,
    	source_field=field, metric=NumericMetric.MEAN)
      

Plan

Use the plan subcommand to preview changes you have defined for your environment before you apply them. This preview shows you any new resources that are created, updated, or deleted. You can iterate between Author and Plan steps until you are satisfied with the changes.

You can invoke the plan subcommand when you want feedback on your work:

validio code plan

• If you are not running the plan command from your project's root directory, use the --directory flag to specify your working directory.
• You can use the --diff flag to control whether you want a summary, a full diff, or a partial diff that only shows the changes to be made.

In the following example, the output from the plan subcommand describes that the program adds 6 resources to the environment:

validio code plan --diff=none

GcpCredential 'example-credential' will be created
GcpBigQuerySource 'example-source' will be created
TumblingWindow 'example-window' will be created
Segmentation 'example-segmentation' will be created
NumericValidator 'mean_of_Age' will be created
NumericValidator 'mean_of_Credit_scoring' will be created

Plan: 6 to create, 0 to update, 0 to delete.

Targeting individual resources

Sometimes you only want to see the plan result or apply changes for selected resources, similar to how Terraform's target flag works. This is possible by using flags named like the resources, e.g. --credential, --source, --validator etc. Each flag can be specified multiple times to target multiple resources.

validio code plan \
  --credential example-credential \
  --validator mean_of_Age \
  --validator mean_of_Credit_scoring

This will only target the three specified resources and ignore any potential changes to the other resources. The flag works with both Plan and Apply.

Apply

Use the apply subcommand to accept and perform the planned changes to your environment. Running apply also performs the Plan step, which gives you a final look at the planned changes.

validio code apply

🚧

Important

Apply has an explicit confirmation step because it is a potentially destructive operation.

To skip the confirmation prompt and apply the planned changes immediately, you can pass the --auto-approve flag. This may be useful in CI/CD workflows where you want to apply changes that are automatically merged to a branch.

he following example shows the output from plan, followed by a request for confirmation, and finally a confirmation message when apply completes.

validio code apply --diff=none

GcpCredential 'example-credential' will be created
GcpBigQuerySource 'example-source' will be created
TumblingWindow 'example-window' will be created
Segmentation 'example-segmentation' will be created
NumericValidator 'mean_of_Age' will be created
NumericValidator 'mean_ofCredit_scoring' will be created

Plan: 6 to create, 0 to update, 0 to delete.

Do you want to perform these operations?
        Only 'yes' will be accepted to approve
Enter a value: yes

...
Apply complete! Resources: 6 created, 0 updated, 0 deleted