HomeRequest DemoContact
Back to All

Changelog 1.8

November 21st, 2023 by Lars Fredholm

Customization features

If there is one word to really sum up this release it is customization. We have added lots of capabilities to help you customize the way you validate your data and route information about incidents to the relevant stakeholders.

Databricks

We have added support for Databricks as our newest data source integration!

Custom SQL validator

We have created a Custom SQL validator. Now you can leverage the power of SQL to create any custom validation of your data. This allow you to not only take advantage of SQL warehouse-specific functionality, but you can also include templates into those queries to fully utilize our segmentation and windowing logic. To help you get started, check out the list of custom SQL recipes.

SQL filter

There is now a SQL filter which allows you to create advanced filters using SQL. The SQL filter works for all Source types, including non-SQL data sources. This allows you to easily combine multiple conditions together in order to filter the data that goes into a validator.

Validator recipes

We have added a new section of Validator recipes to highlight many of the different ways that our existing validators can be used.

Incident severity

We have added a notion of incident severity – basically how extreme an incident is. All validator incidents will now be tagged with a low, medium, or high severity so that you can prioritize and only be notified about the most critical incidents.

User-defined tagging

We have added the ability to tag catalog assets, sources, and validators with user-defined key-value pairs. Utilize this information with our improved Rules to have even more fine-grained control over the notification routing.

Improved notification rules

Our Rules now allow you to use even more information when deciding if and how you want to route incidents. This includes conditions that you compose together based on:

  • Source
  • Source owner
  • Incident type
  • Incident severity
  • Segment values
  • User-defined tags

1.8.5

  • Change: Change sorting behavior for name fields (UI-2806)
  • Change: Improve lineage collection for Databricks (VR-2640)
  • Change: Extend SQL filter with support for UPPER and LOWER functions (VR-2714)
  • Change: Default sensitivity for dynamic threshold in recommendations changed from 3 to 2 (VR-2599)
  • Bug Fix: Fix race-condition when batch inserting incidents (UI-2842)
  • Bug Fix: Fix issue with rules config after changing channel (UI-2846)

1.8.4

  • Bug Fix: Use native browser validation for email-fields (UI-2831)
  • Bug Fix: Change "Back to segmentations"-link (UI-2812)

1.8.3

API additions

  • Added methods for updating existing conditions on notification rules
  • Added stats-field to Credential types with count of sources and catalog assets that are related to the credential

Other improvements & fixes

  • Feature: Add ability to update notification rule conditions (UI-2769, UI-276)
  • Feature: Add config box to notification rule detail (UI-2758)
  • Feature: Add TimeRangeControl to Incident details (UI-2770, UI-2770)
  • Feature: Add search to schema views (UI-2768)
  • Feature: Add search to recommended validators (UI-2767)
  • Feature: Add severity info in slack & webhook notifications (UI-2752, UI-2752)
  • Feature: Add back button to validator wizards (UI-2772)
  • Feature: Improve categorical distribution validator debug query, returns added, changed or removed categories (VR-2746)
  • Bug Fix: Source errors not reported in overview page recent activity (UI-2777, UI-2783)
  • Bug Fix: Display correct percentage for backfill progress (UI-2779, UI-2779)
  • Bug Fix: Better error message for sql validator (UI-2776)
  • Bug Fix: Fix layout of Threshold dialog (UI-2737)
  • Bug Fix: Remove update from global windows (UI-2764)
  • Bug Fix: Make table-layout more responsive (UI-2738)
  • Bug fix: Fix issues with state of list filters (UI-2583)
  • Bug fix: Fix layout in configuration banner (UI-2774)
  • Bug fix: Fix numerical anomaly validator float precision handling on snowflake warehouse (VR-2774)
  • Change: Display confirmation when dismissing recommendations (UI-2790)

1.8.2

  • Fix issues of Slack notifications (UI-2766)

1.8.1

  • Fix for issue with platform metrics endpoint (UI-2765)

1.8.0

API changes and additions

  • Field notificationTypenames has been deprecated on object type NotificationRule.
  • Field sources has been deprecated on object type NotificationRule.
  • Input field notificationTypenames has been deprecated in input object type NotificationRuleCreateInput.
  • Input field sources has been deprecated in input object type NotificationRuleCreateInput.
  • Input field notificationTypenames has been deprecated in input object type NotificationRuleUpdateInput.
  • Input field sources has been deprecated in input object type NotificationRuleUpdateInput.

Other improvements & fixes

  • Bug Fix: Use microsecond timestamp precision in debug SQL queries (VR-2703).
  • Bug Fix: Set cursor lower bound in streaming warehouse SQL queries to avoid table scans (VR-2747).
  • Bug Fix: Remove unused pipelines leading to extra SQL queries (VR-2757).
  • Bug Fix: Limit overview statistics to one year, including only existing validators (UI-2664).
  • Feature: Add tumbling window polling without a delay (PRO-612).
  • Feature: Allow schema changes in source edit and reinferring source schema (PRO-597).
  • Feature: Add schema change detection for nested fields (VR-2697)