VPC Deployment Installation

Install Validio on customer VPC. The Validio application is distributed as a Helm chart. The Helm chart is installable on managed Kubernetes clusters.

Validio is available for the following cloud providers:

  • Google Cloud Platform (GCP): GKE.
  • Amazon Web Services (AWS): EKS.
  • Microsoft Azure: AKS.

Prerequisites

📘

Note

Installing Validio on Customer VPC requires knowledge of:

  • Kubernetes: how to install, configure, and monitor resources on AWS, GCP or Azure.
  • Helm: how to install and configure helm charts on Kubernetes.

1. Access to docker images using a JSON file provided by Validio. For more information,contact us.

2. Configure a Kubernetes cluster on GCP, AWS or Azure. Considering the following:

  1. All nodes must be in the same AWS availability zone, GCP zone or Azure zone.
  2. Persistent volume provisioning enabled, to create Kubernetes PVC resources.
  3. We also recommended a configured load balance or ingress controller, such as:
    1. GCP Ingress Controller
    2. AWS Load Balancer Controller
    3. AKS Application routing add-on

3. Install the following:


Installation

1. Create namespace

Create the namespace you want to use for your configuration. In our examples, we use validio.

kubectl create namespace validio

2. Add the docker-registry secret

Create a Kubernetes Secret to pull the docker images required by the helm chart. This requires the JSON file provided by Validio.

First, set the VALIDIO_SECRET environment variable to point to your JSON file:

export VALIDIO_SECRET=/home/user/Downloads/validio-secret.json

Then, add the Kubernetes Secret:

kubectl -n validio create secret docker-registry artifact-registry --docker-server=https://europe-docker.pkg.dev --docker-username=_json_key --docker-password="$(cat ${VALIDIO_SECRET})" 

3. Install the helm chart

📘

The helm chart installation requires the validio-values.yaml file. For details, refer to configuration and examples.

helm install validio oci://europe-docker.pkg.dev/validio-platform-prod/charts/validio --version [your-validio-version] --namespace validio --values validio-values.yaml

4. Upgrade Validio version

📘

Note

Make sure you use the correct validio-values.yaml file when upgrading Validio. Otherwise, you risk changing your configuration.

helm upgrade validio oci://europe-docker.pkg.dev/validio-platform-prod/charts/validio --version [your-validio-version] --namespace validio --values validio-values.yaml

5. Configure the validio-values.yaml file

The Helm chart supports the following keys in the validio-values.yaml file:

KeyDefault valueDescription
meilisearch.resources.*nilResources for Meilisearch service. Suggested values: memory 1000Mi, cpu 200m.
meilisearch.storage.capacitynilStorage capacity for Meilisearch service. Suggested value: 10Gi
meilisearch.storage.classnilStorage class for Meilisearch service. For example, premium-rwo for GCP and gp2 for AWS. Leave empty to use cloud provider default.
otel-collector.config.customernilName of Customer, decide this value together with Validio
otel-collector.env.http_proxynilSet proxy forHTTP requests
otel-collector.env.https_proxynilSet proxy forHTTPS requests
postgres.resources.*nilResources for Postgres service . Suggested values: memory 4Gi and cpu 1000m.
postgres.storage.capacitynilStorage capacity for Postgres service. Suggested value: 50Gi.
postgres.storage.classnilStorage class for Postgres service. For example, premium-rwo for GCP and gp2 for AWS. Leave empty to use cloud provider default.
redis.resources.*nilResources for Postgres service. Suggested values: memory 500Mi and cpu 200m.
redis.storage.capacitynilStorage capacity for Redis service. Suggested values: 20Gi.
redis.storage.classnilStorage class for Redis service. For example, premium-rwo for GCP and gp2 for AWS. Leave empty to use cloud provider default.
seastar.env.http_proxynilSet proxy forHTTP requests, will be used by Sentry
seastar.env.https_proxynilSet proxy forHTTPS requests, will be used by Sentry
seastar.probes.*nilControl the Kubernetes Readiness and Liveness probes.
seastar.resources.*nilResources for Seastar service . Suggested values: memory 1000Mi, cpu 200m.
surface.env.http_proxynilSet proxy forHTTP requests, will be used by Sentry
surface.env.https_proxynilSet proxy forHTTPS requests, will be used by Sentry
surface.ingress.enabledfalseSet to true to enable Kubernetes ingress. Note: This requires an ingress controller and that all ingress parameters are set accordingly.
surface.ingress.hostnilThe ingress hostname. Note: You must also set a DNS name.
surface.ingress.path/Set the ingress path.
surface.ingress.pathTypePrefixSet the Ingress path type.
surface.ingress.annotationsnilSet the ingress annotations.
surface.ingress.managedCertfalseCreate a ManagedCertificate resource for the ingress. Note: For GCP only.
surface.ingress.tls.enablefalseEnable Ingress TLS, required for use in Azure with cert-manager
surface.probes.*nilControl the Kubernetes Readiness and Liveness probes.
surface.resources.*nilResources for Surface service. Suggested values: memory 1000Mi, cpu 200m.
tags.metricsnilSet to true to enable metrics collection using OpenTelemetry Collector
ve.diver.certificate.enabledfalseSet to true to enable custom self signed certificates for outbound communication
ve.diver.certificate.contentnilSelf-signed certificate content, including header and footer.
ve.diver.env.http_proxynilSet proxy for HTTP requests when sending notifications to Slack, Microsoft Teams, and Webhook.
ve.diver.env.https_proxynilSet proxy for HTTPS requests when sending notifications to Slack, Microsoft Teams, and Webhook.
ve.diver.probes.*nilControl the Kubernetes Readiness and Liveness probes.
ve.diver.resources.*nilResources for Diver service. Suggested values: memory 500Mi, cpu 200m.
ve.ingress.probes.*nilControl the Kubernetes Readiness and Liveness probes.
ve.ingress.resources.*nilResources for Ingress service. Suggested vlaues: memory 1000Mi, cpu 500m. Note: This is not a Kubernetes ingress parameter.
ve.ingress.cache.size1GiLocal disk cache for Ingress service. Will be allocated on the host where the pod runs.
ve.pipelines.probes.*nilControl the Kubernetes Readiness and Liveness probes.
ve.pipelines.resources.*nilResources for Pipelines service. Suggested values: memory 2Gi, cpu 1000m.
ve.pipelines.cache.size1GiLocal disk cache for Pipelines service. Will be allocated on the host where the pod runs.

If the default value is false or true, the value type is bool. For all others, the value type is String.

All <service>.resources.* keys supports both CPU and Memory resources, specified only as <service>.resource.*.

Available keys for each service:

<service>:
  resources:
    limits:
      cpu: <value>
      memory: <value>
    requests:
      cpu: <value>
      memory: <value>
📘

Note

The following are memory and CPU recommendations:

  • Set a limit for memory use with <service>.resources.limits.memory.
  • Only set requests for CPU resources with <service>.resources.requests.cpu.

For more information, refer to Kubernetes resources documentation.

If you have special requirements for Kuberneres pod placements, like using affinity, tolerations or nodeSelector that is possible on all our services. Here is an example of how it would be done:

<service>:
  nodeSelector:
    <label>: "<value>"
  tolerations:
    - key: <key>
      operator: Equal
      value: "<value>"
      effect: NoSchedule

Most services also supports controlling the Kubernetes Liveness and Readiness probes. They are specified above in the table as <service>.probes.*. Available keys are listed below, note that they are enabled by default and the commented values are the default values, so no need to make any changes if you are happy with the default values.

<service>:
  probes:
    liveness:
      enabled: true
      # failureThreshold: 3
      # initialDelaySeconds: 0
      # periodSeconds: 10
      # successThreshold: 1
      # terminationGracePeriodSeconds: 30
      # timeoutSeconds: 1
    readiness:
      enabled: true
      # failureThreshold: 3
      # initialDelaySeconds: 0
      # periodSeconds: 10
      # successThreshold: 1
      # terminationGracePeriodSeconds: 30
      # timeoutSeconds: 1

Sign in to the Validio platform

The first time you sign in to the Validio platform, you must use the admin credentials created in the Kubernetes Secret.

🚧

Caution

Do not change or update anything in the Kubernetes Secret. You can change the password in the Validio platform.

kubectl -n validio get secrets validio-postgres -o=jsonpath='{.data.validio_app_auth_password}' | base64 -d ; echo

Access your Validio platform:

In your browser, navigate to the hostname specified in the surface.ingress.host key. For example, https://validio.example.com/

If you did not configure an ingress, use port-forward to forward the Validio platform to your localhost:

kubectl -n validio port-forward svc/validio-surface 8889

Then, you can find it on http://localhost:8889/


Deployment Examples

Example of GCP ingress

surface:
  ingress:
    enabled: true
    host: validio.example.com
    managedCert: true
    annotations:
      kubernetes.io/ingress.allow-http: "false"
      kubernetes.io/ingress.class: gce

This example creates a Kubernetes Ingress resource and a ManagedCertificate resource, with the specified hostname. Make sure that the created DNS A record points to the IP address of the load balancer. Without a DNS A record, the certificate is not created and the Kubernetes Ingress won’t work.

Use the following command to verify the IP address of the load balance:

kubectl -n validio get ingress validio-surface -o 
jsonpath='{.status.loadBalancer.ingress[0].ip}'

Example of AWS ingress

surface:
  ingress:
    enabled: true
    host: validio.example.com
    annotations:
      alb.ingress.kubernetes.io/certificate-arn: <ARN of certificate>
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/success-codes: 200,302
      alb.ingress.kubernetes.io/target-type: ip
      kubernetes.io/ingress.class: alb

This requires the AWS Load Balancer Controller Add-On. This example creates a Kubernetes Ingress resource with the specified hostname. You must first create a certificate with AWS Certificate Manager as specified in the annotations part. Then, use the Alias option to create a DNS A record that points to the endpoint of the load balancer.

Use the following command to verify the endpoint of the load balancer:

kubectl -n validio get ingress validio-surface -o 
jsonpath='{.status.loadBalancer.ingress[0].hostname}'

Example of Azure ingress

surface:
  ingress:
    enabled: true
    host: validio.example.com
    annotations:
      cert-manager.io/acme-challenge-type: http01
      cert-manager.io/cluster-issuer: letsencrypt
    ingressClassName: webapprouting.kubernetes.azure.com
    tls:
      enabled: true

This requires the AKS Application routing add-on with a workin external-dns setup and cert-manager for TLS . This example creates a Kubernetes Ingress resource with the specified hostname using TLS.

Example of Nginx Ingress

Configuring an Nginx Ingress can be done in a few different ways, here we will show just the basic options needed. For your environment you might need to add specific annotations local to your Kubernetes installation.

surface:
  ingress:
    enabled: true
    host: validio.example.com

Example of default memory and cpu resources set

🚧

Caution

Below is an example configuration using the above suggested resource configuration and some other sensible values. The YAML key surface.ingressbelow are GCP, AWS or Azure specific, please refer to examples above. Validio will provide customer specific configuration when applicable

meilisearch:
  resources:
    limits:
      memory: 1000Mi
    requests:
      cpu: 200m
  storage:
    capacity: 10Gi
    class: managed-premium
postgres:
  resources:
    limits:
      memory: 4000Mi
    requests:
      cpu: 200m
  storage:
    capacity: 50Gi
    class: managed-premium
redis:
  resources:
    limits:
      memory: 500Mi
    requests:
      cpu: 200m
  storage:
    capacity: 20Gi
    class: managed-premium
seastar:
  resources:
    limits:
      memory: 1000Mi
    requests:
      cpu: 200m
surface:
  ingress:
    enabled: true
    host: example.validio.io
  resources:
    limits:
      memory: 1000Mi
    requests:
      cpu: 200m
  service:
    type: ClusterIP
ve:
  diver:
    resources:
      limits:
        memory: 500Mi
      requests:
        cpu: 200m
  ingress:
    cache:
      size: 10G
    resources:
      limits:
        memory: 1000Mi
      requests:
        cpu: 200m
  pipelines:
    cache:
      size: 10G
    resources:
      limits:
        memory: 1000Mi
      requests:
        cpu: 200m