Skip to content

column monthly monitoring checks

ColumnNumericMonthlyMonitoringChecksSpec

Container of built-in preconfigured data quality monitoring on a column level that are checking numeric values at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_negative_count Verifies that the number of negative values in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNegativeCountCheckSpec
monthly_negative_percent Verifies that the percentage of negative values in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNegativePercentCheckSpec
monthly_non_negative_count Verifies that the number of non-negative values in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNonNegativeCountCheckSpec
monthly_non_negative_percent Verifies that the percentage of non-negative values in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNonNegativePercentCheckSpec
monthly_expected_numbers_in_use_count Verifies that the expected numeric values were found in the column. Raises a data quality issue when too many expected values were not found (were missing). Stores the most recent row count for each month when the data quality check was evaluated. ColumnExpectedNumbersInUseCountCheckSpec
monthly_number_value_in_set_percent The check measures the percentage of rows whose value in a tested column is one of values from a list of expected values or the column value is null. Verifies that the percentage of rows having a valid column value does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNumberValueInSetPercentCheckSpec
monthly_values_in_range_numeric_percent Verifies that the percentage of values from range in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnValuesInRangeNumericPercentCheckSpec
monthly_values_in_range_integers_percent Verifies that the percentage of values from range in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnValuesInRangeIntegersPercentCheckSpec
monthly_value_below_min_value_count The check counts the number of values in the column that is below the value defined by the user as a parameter. Stores the most recent row count for each month when the data quality check was evaluated. ColumnValueBelowMinValueCountCheckSpec
monthly_value_below_min_value_percent The check counts the percentage of values in the column that is below the value defined by the user as a parameter. Stores the most recent row count for each month when the data quality check was evaluated. ColumnValueBelowMinValuePercentCheckSpec
monthly_value_above_max_value_count The check counts the number of values in the column that is above the value defined by the user as a parameter. Stores the most recent row count for each month when the data quality check was evaluated. ColumnValueAboveMaxValueCountCheckSpec
monthly_value_above_max_value_percent The check counts the percentage of values in the column that is above the value defined by the user as a parameter. Stores the most recent row count for each month when the data quality check was evaluated. ColumnValueAboveMaxValuePercentCheckSpec
monthly_max_in_range Verifies that the maximal value in a column does not exceed the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnMaxInRangeCheckSpec
monthly_min_in_range Verifies that the minimal value in a column does not exceed the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnMinInRangeCheckSpec
monthly_mean_in_range Verifies that the average (mean) of all values in a column does not exceed the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnMeanInRangeCheckSpec
monthly_percentile_in_range Verifies that the percentile of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPercentileInRangeCheckSpec
monthly_median_in_range Verifies that the median of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnMedianInRangeCheckSpec
monthly_percentile_10_in_range Verifies that the percentile 10 of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPercentile10InRangeCheckSpec
monthly_percentile_25_in_range Verifies that the percentile 25 of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPercentile25InRangeCheckSpec
monthly_percentile_75_in_range Verifies that the percentile 75 of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPercentile75InRangeCheckSpec
monthly_percentile_90_in_range Verifies that the percentile 90 of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPercentile90InRangeCheckSpec
monthly_sample_stddev_in_range Verifies that the sample standard deviation of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnSampleStddevInRangeCheckSpec
monthly_population_stddev_in_range Verifies that the population standard deviation of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPopulationStddevInRangeCheckSpec
monthly_sample_variance_in_range Verifies that the sample variance of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnSampleVarianceInRangeCheckSpec
monthly_population_variance_in_range Verifies that the population variance of all values in a column is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPopulationVarianceInRangeCheckSpec
monthly_sum_in_range Verifies that the sum of all values in a column does not exceed the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnSumInRangeCheckSpec
monthly_invalid_latitude_count Verifies that the number of invalid latitude values in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnInvalidLatitudeCountCheckSpec
monthly_valid_latitude_percent Verifies that the percentage of valid latitude values in a column does not fall below the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnValidLatitudePercentCheckSpec
monthly_invalid_longitude_count Verifies that the number of invalid longitude values in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnInvalidLongitudeCountCheckSpec
monthly_valid_longitude_percent Verifies that the percentage of valid longitude values in a column does not fall below the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnValidLongitudePercentCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnSchemaMonthlyMonitoringChecksSpec

Container of built-in preconfigured data quality checks on a column level that are checking the column schema at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_column_exists Checks the metadata of the monitored table and verifies if the column exists. Stores the most recent value for each month when the data quality check was evaluated. ColumnSchemaColumnExistsCheckSpec
monthly_column_type_changed Checks the metadata of the monitored column and detects if the data type (including the length, precision, scale, nullability) has changed since the last month. Stores the most recent hash for each month when the data quality check was evaluated. ColumnSchemaTypeChangedCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnIntegrityMonthlyMonitoringChecksSpec

Container of integrity data quality monitoring checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_foreign_key_not_match_count Verifies that the number of values in a column that does not match values in another table column does not exceed the set count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnIntegrityForeignKeyNotMatchCountCheckSpec
monthly_foreign_key_match_percent Verifies that the percentage of values in a column that matches values in another table column does not exceed the set count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnIntegrityForeignKeyMatchPercentCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnDatetimeMonthlyMonitoringChecksSpec

Container of date-time data quality monitoring checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_date_match_format_percent Verifies that the percentage of date values matching the given format in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each monthly monitoring. ColumnDatetimeDateMatchFormatPercentCheckSpec
monthly_date_values_in_future_percent Verifies that the percentage of date values in future in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnDateValuesInFuturePercentCheckSpec
monthly_datetime_value_in_range_date_percent Verifies that the percentage of date values in the range defined by the user in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnDatetimeValueInRangeDatePercentCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnSqlMonthlyMonitoringChecksSpec

Container of built-in preconfigured data quality checks on a column level that are using custom SQL expressions (conditions).

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_sql_condition_passed_percent_on_column Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Stores the most recent row count for each month when the data quality check was evaluated. ColumnSqlConditionPassedPercentCheckSpec
monthly_sql_condition_failed_count_on_column Verifies that a number of rows failed a custom SQL condition(expression) does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnSqlConditionFailedCountCheckSpec
monthly_sql_aggregate_expr_column Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the set range. Stores the most recent row count for each month when the data quality check was evaluated. ColumnSqlAggregateExprCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnMonthlyMonitoringCheckCategoriesSpec

Container of column level monthly monitoring checks. Contains categories of monthly monitoring checks.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
nulls Monthly monitoring checks of nulls in the column ColumnNullsMonthlyMonitoringChecksSpec
numeric Monthly monitoring checks of numeric in the column ColumnNumericMonthlyMonitoringChecksSpec
strings Monthly monitoring checks of strings in the column ColumnStringsMonthlyMonitoringChecksSpec
uniqueness Monthly monitoring checks of uniqueness in the column ColumnUniquenessMonthlyMonitoringChecksSpec
datetime Monthly monitoring checks of datetime in the column ColumnDatetimeMonthlyMonitoringChecksSpec
pii Monthly monitoring checks of Personal Identifiable Information (PII) in the column ColumnPiiMonthlyMonitoringChecksSpec
sql Monthly monitoring checks of custom SQL checks in the column ColumnSqlMonthlyMonitoringChecksSpec
bool Monthly monitoring checks of booleans in the column ColumnBoolMonthlyMonitoringChecksSpec
integrity Monthly monitoring checks of integrity in the column ColumnIntegrityMonthlyMonitoringChecksSpec
accuracy Monthly monitoring checks of accuracy in the column ColumnAccuracyMonthlyMonitoringChecksSpec
datatype Monthly monitoring checks of datatype in the column ColumnDatatypeMonthlyMonitoringChecksSpec
anomaly Monthly monitoring checks of anomaly in the column ColumnAnomalyMonthlyMonitoringChecksSpec
schema Monthly monitoring column schema checks ColumnSchemaMonthlyMonitoringChecksSpec
comparisons Dictionary of configuration of checks for table comparisons at a column level. The key that identifies each comparison must match the name of a data comparison that is configured on the parent table. ColumnComparisonMonthlyMonitoringChecksSpecMap
custom Dictionary of custom checks. The keys are check names within this category. CustomCheckSpecMap

ColumnUniquenessMonthlyMonitoringChecksSpec

Container of uniqueness data quality monitoring checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_distinct_count Verifies that the number of distinct values in a column does not fall below the minimum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnDistinctCountCheckSpec
monthly_distinct_percent Verifies that the percentage of distinct values in a column does not fall below the minimum accepted percent. Stores the most recent row count for each month when the data quality check was evaluated. ColumnDistinctPercentCheckSpec
monthly_duplicate_count Verifies that the number of duplicate values in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnDuplicateCountCheckSpec
monthly_duplicate_percent Verifies that the percentage of duplicate values in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnDuplicatePercentCheckSpec
monthly_anomaly_differencing_distinct_count_30_days Verifies that the distinct count in a monitored column is within a two-tailed percentile from measurements made during the last 30 days. ColumnAnomalyDifferencingDistinctCount30DaysCheckSpec
monthly_anomaly_differencing_distinct_count Verifies that the distinct count in a monitored column is within a two-tailed percentile from measurements made during the last 90 days. ColumnAnomalyDifferencingDistinctCountCheckSpec
monthly_anomaly_stationary_distinct_percent_30_days Verifies that the distinct percent in a monitored column is within a two-tailed percentile from measurements made during the last 30 days. ColumnAnomalyStationaryDistinctPercent30DaysCheckSpec
monthly_anomaly_stationary_distinct_percent Verifies that the distinct percent in a monitored column is within a two-tailed percentile from measurements made during the last 90 days. ColumnAnomalyStationaryDistinctPercentCheckSpec
monthly_change_distinct_count Verifies that the distinct count in a monitored column has changed by a fixed rate since the last readout. ColumnChangeDistinctCountCheckSpec
monthly_change_distinct_count_since_7_days Verifies that the distinct count in a monitored column has changed by a fixed rate since the last readout from last week. ColumnChangeDistinctCountSince7DaysCheckSpec
monthly_change_distinct_count_since_30_days Verifies that the distinct count in a monitored column has changed by a fixed rate since the last readout from last month. ColumnChangeDistinctCountSince30DaysCheckSpec
monthly_change_distinct_count_since_yesterday Verifies that the distinct count in a monitored column has changed by a fixed rate since the last readout from yesterday. ColumnChangeDistinctCountSinceYesterdayCheckSpec
monthly_change_distinct_percent Verifies that the distinct percent in a monitored column has changed by a fixed rate since the last readout. ColumnChangeDistinctPercentCheckSpec
monthly_change_distinct_percent_since_7_days Verifies that the distinct percent in a monitored column has changed by a fixed rate since the last readout from last week. ColumnChangeDistinctPercentSince7DaysCheckSpec
monthly_change_distinct_percent_since_30_days Verifies that the distinct percent in a monitored column has changed by a fixed rate since the last readout from last month. ColumnChangeDistinctPercentSince30DaysCheckSpec
monthly_change_distinct_percent_since_yesterday Verifies that the distinct percent in a monitored column has changed by a fixed rate since the last readout from yesterday. ColumnChangeDistinctPercentSinceYesterdayCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnAnomalyMonthlyMonitoringChecksSpec

Container of built-in preconfigured data quality checks on a column level for detecting anomalies.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_mean_change Verifies that the mean value in a column changed in a fixed rate since last readout. ColumnChangeMeanCheckSpec
monthly_median_change Verifies that the median in a column changed in a fixed rate since last readout. ColumnChangeMedianCheckSpec
monthly_sum_change Verifies that the sum in a column changed in a fixed rate since last readout. ColumnChangeSumCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnComparisonMonthlyMonitoringChecksSpec

Container of built-in preconfigured column level comparison checks that compare min/max/sum/mean/nulls measures between the column in the tested (parent) table and a matching reference column in the reference table (the source of truth). This is the configuration for monthly monitoring checks that are counted in KPIs.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_sum_match Verifies that percentage of the difference between the sum of values in a tested column in a parent table and the sum of a values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. ColumnComparisonSumMatchCheckSpec
monthly_min_match Verifies that percentage of the difference between the minimum value in a tested column in a parent table and the minimum value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. ColumnComparisonMinMatchCheckSpec
monthly_max_match Verifies that percentage of the difference between the maximum value in a tested column in a parent table and the maximum value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. ColumnComparisonMaxMatchCheckSpec
monthly_mean_match Verifies that percentage of the difference between the mean (average) value in a tested column in a parent table and the mean (average) value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. ColumnComparisonMeanMatchCheckSpec
monthly_not_null_count_match Verifies that percentage of the difference between the count of not null values in a tested column in a parent table and the count of not null values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. ColumnComparisonNotNullCountMatchCheckSpec
monthly_null_count_match Verifies that percentage of the difference between the count of null values in a tested column in a parent table and the count of null values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. ColumnComparisonNullCountMatchCheckSpec
reference_column The name of the reference column name in the reference table. It is the column to which the current column is compared to. string
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnBoolMonthlyMonitoringChecksSpec

Container of boolean monitoring data quality checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_true_percent Verifies that the percentage of true values in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnTruePercentCheckSpec
monthly_false_percent Verifies that the percentage of false values in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnFalsePercentCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnNullsMonthlyMonitoringChecksSpec

Container of nulls data quality monitoring checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_nulls_count Verifies that the number of null values in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNullsCountCheckSpec
monthly_nulls_percent Verifies that the percentage of null values in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNullsPercentCheckSpec
monthly_not_nulls_count Verifies that the number of not null values in a column does not fall below the minimum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNotNullsCountCheckSpec
monthly_not_nulls_percent Verifies that the percentage of not nulls in a column does not fall below the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnNotNullsPercentCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnDatatypeMonthlyMonitoringChecksSpec

Container of datatype data quality monitoring checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_string_datatype_detected Detects the data type of text values stored in the column. The sensor returns the code of the detected type of column data: 1 - integers, 2 - floats, 3 - dates, 4 - timestamps, 5 - booleans, 6 - strings, 7 - mixed data types. Raises a data quality issue when the detected data type does not match the expected data type. Stores the most recent row count for each month when the data quality check was evaluated. ColumnDatatypeStringDatatypeDetectedCheckSpec
monthly_string_datatype_changed Detects that the data type of texts stored in a text column has changed since the last verification. The sensor returns the detected type of column data: 1 - integers, 2 - floats, 3 - dates, 4 - timestamps, 5 - booleans, 6 - strings, 7 - mixed data types. Stores the most recent captured value for each day when the data quality check was evaluated. ColumnDatatypeStringDatatypeChangedCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnPiiMonthlyMonitoringChecksSpec

Container of PII data quality monitoring checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_contains_usa_phone_percent Verifies that the percentage of rows that contains a USA phone number in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPiiContainsUsaPhonePercentCheckSpec
monthly_contains_usa_zipcode_percent Verifies that the percentage of rows that contains a USA zip code in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPiiContainsUsaZipcodePercentCheckSpec
monthly_contains_email_percent Verifies that the percentage of rows that contains emails in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPiiContainsEmailPercentCheckSpec
monthly_contains_ip4_percent Verifies that the percentage of rows that contains IP4 address values in a column does not fall below the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPiiContainsIp4PercentCheckSpec
monthly_contains_ip6_percent Verifies that the percentage of rows that contains valid IP6 address values in a column does not fall below the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnPiiContainsIp6PercentCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnStringsMonthlyMonitoringChecksSpec

Container of strings data quality monitoring checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_string_max_length Verifies that the length of string in a column does not exceed the maximum accepted length. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringMaxLengthCheckSpec
monthly_string_min_length Verifies that the length of string in a column does not exceed the minimum accepted length. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringMinLengthCheckSpec
monthly_string_mean_length Verifies that the length of string in a column does not exceed the mean accepted length. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringMeanLengthCheckSpec
monthly_string_length_below_min_length_count The check counts those strings with length below the one provided by the user in a column. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringLengthBelowMinLengthCountCheckSpec
monthly_string_length_below_min_length_percent The check counts percentage of those strings with length below the one provided by the user in a column. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringLengthBelowMinLengthPercentCheckSpec
monthly_string_length_above_max_length_count The check counts those strings with length above the one provided by the user in a column. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringLengthAboveMaxLengthCountCheckSpec
monthly_string_length_above_max_length_percent The check counts percentage of those strings with length above the one provided by the user in a column. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringLengthAboveMaxLengthPercentCheckSpec
monthly_string_length_in_range_percent The check counts percentage of those strings with length in the range provided by the user in a column. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringLengthInRangePercentCheckSpec
monthly_string_empty_count Verifies that the number of empty strings in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringEmptyCountCheckSpec
monthly_string_empty_percent Verifies that the percentage of empty strings in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringEmptyPercentCheckSpec
monthly_string_valid_dates_percent Verifies that the percentage of valid dates in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringValidDatesPercentCheckSpec
monthly_string_whitespace_count Verifies that the number of whitespace strings in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringWhitespaceCountCheckSpec
monthly_string_whitespace_percent Verifies that the percentage of whitespace strings in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringWhitespacePercentCheckSpec
monthly_string_surrounded_by_whitespace_count Verifies that the number of strings surrounded by whitespace in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringSurroundedByWhitespaceCountCheckSpec
monthly_string_surrounded_by_whitespace_percent Verifies that the percentage of strings surrounded by whitespace in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringSurroundedByWhitespacePercentCheckSpec
monthly_string_null_placeholder_count Verifies that the number of null placeholders in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringNullPlaceholderCountCheckSpec
monthly_string_null_placeholder_percent Verifies that the percentage of null placeholders in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringNullPlaceholderPercentCheckSpec
monthly_string_boolean_placeholder_percent Verifies that the percentage of boolean placeholder for strings in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringBooleanPlaceholderPercentCheckSpec
monthly_string_parsable_to_integer_percent Verifies that the percentage of parsable to integer string in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringParsableToIntegerPercentCheckSpec
monthly_string_parsable_to_float_percent Verifies that the percentage of parsable to float string in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringParsableToFloatPercentCheckSpec
monthly_expected_strings_in_use_count Verifies that the expected string values were found in the column. Raises a data quality issue when too many expected values were not found (were missing). Stores the most recent row count for each month when the data quality check was evaluated. ColumnExpectedStringsInUseCountCheckSpec
monthly_string_value_in_set_percent The check measures the percentage of rows whose value in a tested column is one of values from a list of expected values or the column value is null. Verifies that the percentage of rows having a valid column value does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringValueInSetPercentCheckSpec
monthly_string_valid_country_code_percent Verifies that the percentage of valid country code in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringValidCountryCodePercentCheckSpec
monthly_string_valid_currency_code_percent Verifies that the percentage of valid currency code in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringValidCurrencyCodePercentCheckSpec
monthly_string_invalid_email_count Verifies that the number of invalid emails in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringInvalidEmailCountCheckSpec
monthly_string_invalid_uuid_count Verifies that the number of invalid UUID in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringInvalidUuidCountCheckSpec
monthly_string_valid_uuid_percent Verifies that the percentage of valid UUID in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringValidUuidPercentCheckSpec
monthly_string_invalid_ip4_address_count Verifies that the number of invalid IP4 address in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringInvalidIp4AddressCountCheckSpec
monthly_string_invalid_ip6_address_count Verifies that the number of invalid IP6 address in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringInvalidIp6AddressCountCheckSpec
monthly_string_not_match_regex_count Verifies that the number of strings not matching the custom regex in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringNotMatchRegexCountCheckSpec
monthly_string_match_regex_percent Verifies that the percentage of strings matching the custom regex in a column does not exceed the minimum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringMatchRegexPercentCheckSpec
monthly_string_not_match_date_regex_count Verifies that the number of strings not matching the date format regex in a column does not exceed the maximum accepted count. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringNotMatchDateRegexCountCheckSpec
monthly_string_match_date_regex_percent Verifies that the percentage of strings matching the date format regex in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringMatchDateRegexPercentCheckSpec
monthly_string_match_name_regex_percent Verifies that the percentage of strings matching the name regex in a column does not exceed the maximum accepted percentage. Stores the most recent row count for each month when the data quality check was evaluated. ColumnStringMatchNameRegexPercentCheckSpec
monthly_expected_strings_in_top_values_count Verifies that the top X most popular column values contain all values from a list of expected values. Stores the most recent row count for each month when the data quality check was evaluated. ColumnExpectedStringsInTopValuesCountCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnAccuracyMonthlyMonitoringChecksSpec

Container of accuracy data quality monitoring checks on a column level that are checking at a monthly level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
monthly_total_sum_match_percent Verifies that the percentage of difference in total sum of a column in a table and total sum of a column of another table does not exceed the set number. Stores the most recent row count for each month when the data quality check was evaluated. ColumnAccuracyTotalSumMatchPercentCheckSpec
monthly_total_min_match_percent Verifies that the percentage of difference in total min of a column in a table and total min of a column of another table does not exceed the set number. Stores the most recent row count for each month when the data quality check was evaluated. ColumnAccuracyTotalMinMatchPercentCheckSpec
monthly_total_max_match_percent Verifies that the percentage of difference in total max of a column in a table and total max of a column of another table does not exceed the set number. Stores the most recent row count for each month when the data quality check was evaluated. ColumnAccuracyTotalMaxMatchPercentCheckSpec
monthly_total_average_match_percent Verifies that the percentage of difference in total average of a column in a table and total average of a column of another table does not exceed the set number. Stores the most recent row count for each month when the data quality check was evaluated. ColumnAccuracyTotalAverageMatchPercentCheckSpec
monthly_total_not_null_count_match_percent Verifies that the percentage of difference in total not null count of a column in a table and total not null count of a column of another table does not exceed the set number. Stores the most recent row count for each month when the data quality check was evaluated. ColumnAccuracyTotalNotNullCountMatchPercentCheckSpec
custom_checks Dictionary of additional custom checks within this category. The keys are check names defined in the definition section. The sensor parameters and rules should match the type of the configured sensor and rule for the custom check. CustomCategoryCheckSpecMap

ColumnComparisonMonthlyMonitoringChecksSpecMap

Container of comparison checks for each defined data comparison. The name of the key in this dictionary must match a name of a table comparison that is defined on the parent table. Contains configuration of column level comparison checks. Each column level check container also defines the name of the reference column name to which we are comparing.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
self Dict[string, ColumnComparisonMonthlyMonitoringChecksSpec]