Configuring Windows
Use the Window configuration wizard to define a Window in which data is validated in your Source.
To configure a window,
- On the Source you want to configure a window for, navigate to the Windows tab.
- Click + New window.
- Select the Window type that you want to create: Fixed Batch, Global, Tumbling, or File.
Note
File Windows are only supported for Object storage Source types.
- Under Config, specify the required config options for your Window type.
Note
When configuring windows, a time field is used. The format of the time field is converted to UTC in Validio, but the graphs will always display times in your system's timezone.
4.1 Fixed batch window
| Field | Value | Description |
|---|---|---|
| Data-time field | Field name | Identifier for the index field used to configure the Window. |
| Batch size | Numeric | Number of datapoints (rows) of the Window. For example, 256. |
| Segmented batching | True False | If True, each segment gets a separate Window of batch size length. |
4.2 Global window--Global window requires no configuration.
4.3 Tumbling window
| Field | Values | Description |
|---|---|---|
| Data-time field | Field name | The name of the field that references the timestamp associated with each record or row in the data source. |
| Window size | Numeric | Length of the Window in the selected time unit. For example, 1 |
| Unit | Minute Hour Day Week Month | Unit of time to define Window size. |
| Disable window timeout | True False | Set to true if the window should be automatically closed without considering the most recent data-time. |
4.4 File window
| Field | Value | Description |
|---|---|---|
| Data-time field | Field name | Identifier for the field used to configure the Window. |
Note
File window datasets in distribution shift validation--File window datasets are often used in ML use cases to monitor data drifts.
If you use a production training dataset as reference dataset, as new data is collected, you can monitor distribution shift metrics between the reference dataset and the newly collected dataset.
For information on numeric reference metrics, such as relative entropy, refer to the Numeric distribution or Categorical distribution Validator types.
- Create validators
Use the Validator configuration wizard to create Validators with your configured Window.
Updated about 2 months ago