VPC Deployment Installation

The Validio application is distributed as a Helm chart. The Helm chart is installable on managed Kubernetes clusters. Validio is available for the following cloud providers:

Google Cloud Platform (GCP): GKE.
Amazon Web Services (AWS): EKS.
Microsoft Azure: AKS.

Prerequisites

Installing Validio on Customer VPC requires knowledge of:

Kubernetes: how to install, configure, and monitor resources on AWS, GCP or Azure.
Helm: how to install and configure helm charts on Kubernetes.

1. Access to docker images using a JSON file provided by Validio. For more information, contact us. 2. Configure a Kubernetes cluster on GCP, AWS or Azure. Considering the following:

All nodes must be in the same AWS availability zone, GCP zone or Azure zone.
Persistent volume provisioning enabled, to create Kubernetes PVC resources.
We also recommend a configured load balancer or ingress controller, such as:

3. Install the following:

kubectl CLI tool and set the kubectl context to the destination cluster.
Helm version 3.8.0 or later with Helm OCI support.

Installation

Complete the following steps to install your VPC deployment. Also, you can refer to VPC Deployment Examples.

1. Create namespace

Create the namespace you want to use for your configuration. In our examples, we use validio.

Bash

kubectl create namespace validio

2. Add the docker-registry secret

Create a Kubernetes Secret to pull the docker images required by the helm chart. This requires the JSON file provided by Validio. First, set the VALIDIO_SECRET environment variable to point to your JSON file:

Bash

export VALIDIO_SECRET=/home/user/Downloads/validio-secret.json

Then, add the Kubernetes Secret:

Bash

kubectl -n validio create secret docker-registry artifact-registry --docker-server=https://europe-docker.pkg.dev --docker-username=_json_key --docker-password="$(cat ${VALIDIO_SECRET})"

3. Install the helm chart

The helm chart installation requires the validio-values.yaml file. For details, refer to configuration and examples.

Bash

helm install validio oci://europe-docker.pkg.dev/validio-platform-prod/charts/validio --version [your-validio-version] --namespace validio --values validio-values.yaml

4. Configure the `validio-values.yaml` file

The Helm chart supports the following keys in the validio-values.yaml file:

Key	Default value	Description
`postgres.resources.*`	`nil`	Resources for Postgres service . Suggested values: memory limit `4000Mi`, CPU request `200m`.
`postgres.storage.capacity`	`nil`	Storage capacity for Postgres service. Suggested value: `50Gi`.
`postgres.storage.class`	`nil`	Storage class for Postgres service. For example, `premium-rwo` for GCP and `gp2` for AWS. Leave empty to use cloud provider default.
`redis.resources.*`	`nil`	Resources for Redis service. Suggested values: memory limit `250Mi`, CPU request `50m`. Redis is only lightly used unless streaming sources are enabled; increase the memory limit if you use streaming.
`redis.storage.capacity`	`nil`	Storage capacity for Redis service. Suggested values: `20Gi`.
`redis.storage.class`	`nil`	Storage class for Redis service. For example, `premium-rwo` for GCP and `gp2` for AWS. Leave empty to use cloud provider default.
`surface.env.http_proxy`	`nil`	Set proxy for`HTTP` requests, will be used by Sentry
`surface.env.https_proxy`	`nil`	Set proxy for`HTTPS` requests, will be used by Sentry
`surface.ingress.enabled`	`false`	Set to `true` to enable Kubernetes ingress. Note: This requires an ingress controller and that all ingress parameters are set accordingly.
`surface.ingress.host`	`nil`	The ingress hostname. Note: You must also set a DNS name.
`surface.ingress.path`	`/`	Set the ingress path.
`surface.ingress.pathType`	`Prefix`	Set the Ingress path type.
`surface.ingress.annotations`	`nil`	Set the ingress annotations.
`surface.ingress.managedCert`	`false`	Create a `ManagedCertificate` resource for the ingress. Note: For GCP only.
`surface.ingress.tls.enable`	`false`	Enable Ingress TLS, required for use in Azure with cert-manager
`surface.probes.*`	`nil`	Control the Kubernetes Readiness and Liveness probes.
`surface.resources.*`	`nil`	Resources for Surface service. Suggested values: memory limit `1000Mi`, CPU request `200m`.
`ve.diver.certificate.enabled`	`false`	Set to `true` to enable custom self signed certificates for outbound communication
`ve.diver.certificate.content`	`nil`	Self-signed certificate content, including header and footer.
`ve.diver.env.http_proxy`	`nil`	Set proxy for HTTP requests when sending notifications to Slack, Microsoft Teams, and Webhook.
`ve.diver.env.https_proxy`	`nil`	Set proxy for HTTPS requests when sending notifications to Slack, Microsoft Teams, and Webhook.
`ve.diver.probes.*`	`nil`	Control the Kubernetes Readiness and Liveness probes.
`ve.diver.resources.*`	`nil`	Resources for Diver service. Suggested values: memory limit `500Mi`, CPU request `200m`.
`ve.ingress.probes.*`	`nil`	Control the Kubernetes Readiness and Liveness probes.
`ve.ingress.resources.*`	`nil`	Resources for Ingress service. Suggested values: memory limit `1000Mi`, CPU request `200m`. Note: This is not a Kubernetes ingress parameter.
`ve.ingress.cache.size`	`1Gi`	Local disk cache for Ingress service. Will be allocated on the host where the pod runs.
`ve.pipelines.probes.*`	`nil`	Control the Kubernetes Readiness and Liveness probes.
`ve.pipelines.resources.*`	`nil`	Resources for Pipelines service. Suggested values: memory limit `1000Mi`, CPU request `100m`.
`ve.pipelines.cache.size`	`1Gi`	Local disk cache for Pipelines service. Will be allocated on the host where the pod runs.

If the default value is false or true, the value type is bool. For all others, the value type is String.

All <service>.resources.* keys supports both CPU and Memory resources, specified only as <service>.resource.*. Available keys for each service:

YAML

<service>:
  resources:
    limits:
      cpu: <value>
      memory: <value>
    requests:
      cpu: <value>
      memory: <value>

The following are memory and CPU recommendations:

Memory: set a limit with <service>.resources.limits.memory. Kubernetes terminates a container that exceeds its memory limit, so the limit caps how much a service can consume.
CPU: set a request with <service>.resources.requests.cpu and leave the limit unset. The request reserves capacity when the pod is scheduled, and with no limit in place a service can use spare CPU on the node instead of being throttled.
CPU limits: if your platform policy requires one, set <service>.resources.limits.cpu to 1000m. That is five times the suggested request, so it bounds worst-case use without throttling normal load.

For more information, refer to Kubernetes resources documentation.

Each service runs as a single replicaEvery Validio service runs as exactly one replica and can’t be scaled horizontally. The Helm chart has no replica count or autoscaling setting, and you shouldn’t scale the deployments or statefulsets beyond one replica by any other means. To give a service more capacity, raise its <service>.resources.* values instead.

If you have special requirements for Kubernetes pod placements, like using affinity, tolerations or nodeSelector that is possible on all our services. Here is an example of how it would be done:

YAML

<service>:
  nodeSelector:
    <label>: "<value>"
  tolerations:
    - key: <key>
      operator: Equal
      value: "<value>"
      effect: NoSchedule

Most services also supports controlling the Kubernetes Liveness and Readiness probes. They are specified above in the table as <service>.probes.*. Available keys are listed below, note that they are enabled by default and the commented values are the default values, so no need to make any changes if you are happy with the default values.

YAML

<service>:
  probes:
    liveness:
      enabled: true
      # failureThreshold: 3
      # initialDelaySeconds: 0
      # periodSeconds: 10
      # successThreshold: 1
      # terminationGracePeriodSeconds: 30
      # timeoutSeconds: 1
    readiness:
      enabled: true
      # failureThreshold: 3
      # initialDelaySeconds: 0
      # periodSeconds: 10
      # successThreshold: 1
      # terminationGracePeriodSeconds: 30
      # timeoutSeconds: 1

The first time you sign in to the Validio platform, you must use the admin credentials created in the Kubernetes Secret.

CautionDo not change or update anything in the Kubernetes Secret. You can change the password in the Validio platform.

Bash

kubectl -n validio get secrets validio-postgres -o=jsonpath='{.data.validio_app_auth_password}' | base64 -d ; echo

Access your Validio platform: In your browser, navigate to the hostname specified in the surface.ingress.host key. For example, https://validio.example.com/ If you did not configure an ingress, use port-forward to forward the Validio platform to your localhost:

Bash

kubectl -n validio port-forward svc/validio-surface 8889

Then, you can find it on http://localhost:8889/

Introduction

Using Validio

Validio Tutorials

AI Features

Credentials

Sources

Validators

Catalog

Lineage

Incidents & Errors

Notifications

Integrations

Schema & Profiling

Filters

Windows

Segmentation

Thresholds

Access & Settings

Deployment

Security

Other

VPC Deployment Installation

Prerequisites

Installation

1. Create namespace

2. Add the docker-registry secret

3. Install the helm chart

4. Configure the `validio-values.yaml` file

​Prerequisites

​Installation

​1. Create namespace

​2. Add the docker-registry secret

​3. Install the helm chart

​4. Configure the validio-values.yaml file

​Sign in to the Validio platform

Prerequisites

Installation

1. Create namespace

2. Add the docker-registry secret

3. Install the helm chart

4. Configure the `validio-values.yaml` file

Sign in to the Validio platform