Last updated: September 02, 2024
DQOps REST API default_column_check_patterns models reference
The references of all objects used by default_column_check_patterns REST API operations are listed below.
DataTypeCategory
Enumeration of common data type categories of data types. The providers will use this information to answer which of their native data types matches a category. Some sensors (and profilers) cannot operate on some data types.
The structure of this object is described below
| Data type | Enum values |
|---|---|
| string | numeric_integer numeric_decimal numeric_float datetime_timestamp datetime_datetime datetime_date datetime_time text clob json bool binary array other |
TargetColumnPatternSpec
The configuration of a column pattern to match default column checks. Includes also the pattern for the target table.
The structure of this object is described below
| Property name | Description | Data type |
|---|---|---|
column |
The target column name filter. Accepts wildcards in the format: id, , c_*. | string |
data_type |
The target column data type filter. Filters by a physical (database specific) data type name imported from the data source. Accepts wildcards in the format: int, , big*. | string |
data_type_category |
The filter for a target data type category. | DataTypeCategory |
connection |
The data source connection name filter. Accepts wildcards in the format: conn, , conn*. | string |
schema |
The schema name filter. Accepts wildcards in the format: _prod, , pub*. | string |
table |
The table name filter. Accepts wildcards in the format: _customers, , fact_*. | string |
stage |
The table stage filter. Accepts wildcards in the format: _landing, , staging_*. | string |
table_priority |
The maximum table priority (inclusive) for tables that are covered by the default checks. | integer |
label |
The label filter. Accepts wildcards in the format: _customers, , fact_*. The label must be present on the connection or table. | string |
DefaultColumnChecksPatternListModel
The listing model of column-level default check patterns that is returned by the REST API.
The structure of this object is described below
| Property name | Description | Data type |
|---|---|---|
pattern_name |
Pattern name. | string |
priority |
The priority of the pattern. Patterns with lower values are applied before patterns with higher priority values. | integer |
disabled |
Disables this data quality check configuration. The checks will not be activated. | boolean |
description |
The description (documentation) of this data quality check configuration. | string |
target_column |
The filters for the target column. | TargetColumnPatternSpec |
can_edit |
Boolean flag that decides if the current user can update or delete this object. | boolean |
yaml_parsing_error |
Optional parsing error that was captured when parsing the YAML file. This field is null when the YAML file is valid. If an error was captured, this field returns the file parsing error message and the file location. | string |
ColumnMonitoringCheckCategoriesSpec
Container of column level monitoring, divided by the time window (daily, monthly, etc.)
The structure of this object is described below
| Property name | Description | Data type |
|---|---|---|
daily |
Configuration of daily monitoring evaluated at a column level. | ColumnDailyMonitoringCheckCategoriesSpec |
monthly |
Configuration of monthly monitoring evaluated at a column level. | ColumnMonthlyMonitoringCheckCategoriesSpec |
ColumnPartitionedCheckCategoriesSpec
Container of column level partitioned checks, divided by the time window (daily, monthly, etc.)
The structure of this object is described below
| Property name | Description | Data type |
|---|---|---|
daily |
Configuration of day partitioned data quality checks evaluated at a column level. | ColumnDailyPartitionedCheckCategoriesSpec |
monthly |
Configuration of monthly partitioned data quality checks evaluated at a column level. | ColumnMonthlyPartitionedCheckCategoriesSpec |
ColumnDefaultChecksPatternSpec
The default configuration of column-level data quality checks that are enabled as data observability checks to analyze basic measures and detect anomalies on columns. This configuration serves as a data quality policy that defines the data quality checks that are verified on matching columns.
The structure of this object is described below
| Property name | Description | Data type |
|---|---|---|
priority |
The priority of the pattern. Patterns with lower values are applied before patterns with higher priority values. | integer |
disabled |
Disables this data quality check configuration. The checks will not be activated. | boolean |
description |
The description (documentation) of this data quality check configuration. | string |
target |
The target column filter that are filtering the column, table and connection on which the default checks are applied. | TargetColumnPatternSpec |
profiling_checks |
Configuration of data quality profiling checks that are enabled. Pick a check from a category, apply the parameters and rules to enable it. | ColumnProfilingCheckCategoriesSpec |
monitoring_checks |
Configuration of table level monitoring checks. Monitoring checks are data quality checks that are evaluated for each period of time (daily, weekly, monthly, etc.). A monitoring check stores only the most recent data quality check result for each period of time. | ColumnMonitoringCheckCategoriesSpec |
partitioned_checks |
Configuration of table level date/time partitioned checks. Partitioned data quality checks are evaluated for each partition separately, raising separate alerts at a partition level. The table does not need to be physically partitioned by date, it is possible to run data quality checks for each day or month of data separately. | ColumnPartitionedCheckCategoriesSpec |
DefaultColumnChecksPatternModel
Default column-level checks pattern model that is returned by the REST API. Describes a configuration of data quality checks for a named pattern. DQOps applies these checks on columns that match the filter.
The structure of this object is described below
| Property name | Description | Data type |
|---|---|---|
pattern_name |
Pattern name | string |
pattern_spec |
The default checks specification. | ColumnDefaultChecksPatternSpec |
can_edit |
Boolean flag that decides if the current user can update or delete this object. | boolean |
yaml_parsing_error |
Optional parsing error that was captured when parsing the YAML file. This field is null when the YAML file is valid. If an error was captured, this field returns the file parsing error message and the file location. | string |