Skip to content

table monthly partitioned checks

TableMonthlyPartitionedCheckCategoriesSpec

Container of table level monthly partitioned checks. Contains categories of monthly partitioned checks.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
volume Volume monthly partitioned data quality checks that verify the quality of every month of data separately TableVolumeMonthlyPartitionedChecksSpec
timeliness Monthly partitioned timeliness checks TableTimelinessMonthlyPartitionedChecksSpec
sql Custom SQL monthly partitioned data quality checks that verify the quality of every month of data separately TableSqlMonthlyPartitionedChecksSpec
comparisons Dictionary of configuration of checks for table comparisons. The key that identifies each comparison must match the name of a data comparison that is configured on the parent table. TableComparisonMonthlyPartitionedChecksSpecMap
custom Dictionary of custom checks. The keys are check names. CustomCheckSpecMap

TableSqlMonthlyPartitionedChecksSpec

Container of built-in preconfigured data quality checks on a table level that are using custom SQL expressions (conditions).

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_partition_sql_condition_passed_percent_on_table Verifies that a set percentage of rows passed a custom SQL condition (expression). Creates a separate data quality check (and an alert) for each monthly partition. TableSqlConditionPassedPercentCheckSpec
monthly_partition_sql_condition_failed_count_on_table Verifies that a set number of rows failed a custom SQL condition (expression). Creates a separate data quality check (and an alert) for each monthly partition. TableSqlConditionFailedCountCheckSpec
monthly_partition_sql_aggregate_expr_table Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the set range. Creates a separate data quality check (and an alert) for each monthly partition. TableSqlAggregateExprCheckSpec

TableComparisonMonthlyPartitionedChecksSpecMap

Container of comparison checks for each defined data comparison. The name of the key in this dictionary must match a name of a table comparison that is defined on the parent table. Contains the monthly partitioned comparison checks for each configured reference table.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
access_order boolean
size integer
mod_count integer
threshold integer

TableVolumeMonthlyPartitionedChecksSpec

Container of table level monthly partitioned volume data quality checks.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_partition_min_row_count Verifies that the number of rows in a table does not exceed the minimum accepted count. Creates a separate data quality check (and an alert) for each monthly partition. TableRowCountCheckSpec
monthly_partition_row_count_change Verifies that the total row count of the tested table has changed by a fixed rate since the last readout. TableChangeRowCountCheckSpec

TableComparisonMonthlyPartitionedChecksSpec

Container of built-in comparison (accuracy) checks on a table level that are using a defined comparison to identify the reference table and the data grouping configuration. Contains the monthly partitioned comparison checks, comparing each month of data.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_partition_row_count_match Verifies that the row count of the tested (parent) table matches the row count of the reference table, for each monthly partition (grouping rows by the time period, truncated to the month). Compares each group of data with a GROUP BY clause. Stores the most recent captured value for each monthly partition and optionally data groups. TableComparisonRowCountMatchCheckSpec

TableTimelinessMonthlyPartitionedChecksSpec

Container of table level monthly partitioned timeliness data quality checks.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_partition_data_freshness Monthly partitioned check calculating the number of days since the most recent event (freshness) TableDataFreshnessCheckSpec
monthly_partition_data_staleness Monthly partitioned check calculating the time difference in days between the current date and the most recent data data ingestion timestamp (staleness) TableDataStalenessCheckSpec
monthly_partition_data_ingestion_delay Monthly partitioned check calculating the time difference in days between the most recent event timestamp and the most recent ingestion timestamp TableDataIngestionDelayCheckSpec
monthly_partition_reload_lag Monthly partitioned check calculating the longest time a row waited to be load TablePartitionReloadLagCheckSpec