VPC Communication and Services

Configure your network to communicate with Validio platform and services.

External Communication

To be able to send notifications, error information or metrics you need to make sure your network is configured for this. This can be done either directly to the service or through an HTTP proxy.

HTTP Proxy

Validio supports using HTTP proxy for outbound communication for Slack, MS Teams, Sentry and OpenTelemetry Collector. The URLs for both http_proxy and https_proxy can be the same, it's internally in the application that they are used for http respectively https URLs.

Self-signed Certificate for HTTP Proxy

If you need to add a self signed CA certificate to the trust store for HTTP proxy, you can do that using helm values. The content can be one more more CA certficates in PEM format.

Here is an example:

ve:
  diver:
    certificate:
      enabled: true
      content: |
        -----BEGIN CERTIFICATE-----
        MIIF7zCCA9egAwIBAgIUHTNXv08AHjKzxJHm8TwgkeYysnswDQYJKoZIhvcNAQEL
        BQAwgYYxCzAJBgNVBAYTAlhYMRIwEAYDVQQIDAlTdGF0ZU5hbWUxETAPBgNVBAcM
        CENpdHlOYW1lMRQwEgYDVQQKDAtDb21wYW55TmFtZTEbMBkGA1UECwwSQ29tcGFu
        ...
        Wo7+WuQherTT/VjlyrzYAZMkyhDmXGtJsIH4O5RVfj3t0DyGEmbV+I0zC/ljAVVi
        01FJf058xlPH8mE/Wc9c0ijvuvdELfXjgXY/wWmzIW0xZs8HeysXNzg42QJ3zc08
        Qgx6Flg6JNnP3xiiEjylUV+RPP7KF5eqkwU1SMyvQO2a1rk=
        -----END CERTIFICATE-----

Slack

Use the following configuration to enable HTTP proxy for Slack

ve:
  diver:
    env:
      http_proxy: http://proxy.example.com:8080
      https_proxy: http://proxy.example.com:8080

The following destination will be used by Validio to send notifications to Slack, so make sure it's whitelisted in your network or in your HTTP proxy

hooks.slack.com
slack.com

MS Teams

Use the following configuration to enable HTTP proxy for MS Teams

ve:
  diver:
    env:
      http_proxy: http://proxy.example.com:8080
      https_proxy: http://proxy.example.com:8080

The following destination will be used by Validio to send notifications to MS Teams, make sure they are whitelisted in your network or in your HTTP proxy

*.outlook.office.com
login.botframework.com
login.microsoftonline.com
smba.trafficmanager.net

Sentry

Use the following configuration to enable HTTP proxy for Sentry

seastar:
  env:
    http_proxy: http://proxy.example.com:8080
    https_proxy: http://proxy.example.com:8080
surface:
  env:
    http_proxy: http://proxy.example.com:8080
    https_proxy: http://proxy.example.com:8080

Make sure the following destinations are whitelisted in your network or HTTP proxy

sentry.io
o4506020911316992.ingest.sentry.io

OpenTelemetry Collector

To enable the OpenTelemetry collector you need to add a Kubernetes secret provided by Validio, it is the same secret used for pulling the container images.

First, set the VALIDIO_SECRET environment variable to point to your JSON file:

export VALIDIO_SECRET=/home/user/Downloads/validio-secret.json

Then, add the Kubernetes Secret:

kubectl -n validio create secret generic google-credentials --from-file=key.json="${VALIDIO_SECRET}"

Once the secret is created you can enable the OpenTelemetry Collector

📘

Note

Theotel-collector.config.customerkey needs to be configured with a value that Validio will provide.

otel-collector:
  config:
    customer: <value-provided-by-Validio>
tags:
  metrcis: true

If you require an HTTP proxy, here is how to configure that for OpenTelemetry Collector

otel-collector:
  env:
    http_proxy: http://proxy.example.com:8080
    https_proxy: http://proxy.example.com:8080

Make sure the following destinations are whitelisted in your network or HTTP proxy

oauth2.googleapis.com
monitoring.googleapis.com

Validio Services

The Validio platform is made up of the Validation Engine and Surface.

  • The Validation Engine is the backend application and consists of the services: Diver, Ingress, and Pipelines.
  • Surface is the "backend for the frontend" application and also the name of the service that communicates between the frontend and the Validation Engine.

The following table lists all the Validio services. We recommend that you use this list as a reference when you allocate resources to services running in your system.

Validio ServiceDescription
DiverOrchestrator and running background tasks for the Validio application.
IngressConnects to configured sources to read statistics, data, and source metadata.
PipelinesHandles processing of any read data and performs calculations and anomaly detection on data statistics.
SurfaceResponsible for communication between the frontend application in your browser and the Validation Engine.
SeastarUsed for searching and finding information in the frontend, uses Meilisearch as backend.
PostgresGoverns all artifacts and configuration settings.
RedisUsed as a transport layer between different services.
MeilisearchUsed as a search engine for Surface.