Skip to content

column daily partitioned checks

ColumnNullsDailyPartitionedChecksSpec

Container of nulls data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_nulls_count Verifies that the number of null values in a column does not exceed the set count. Creates a separate data quality check (and an alert) for each daily partition. ColumnNullsCountCheckSpec
daily_partition_nulls_percent Verifies that the percentage of null values in a column does not exceed the set percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnNullsPercentCheckSpec
daily_partition_nulls_percent_anomaly_stationary_30_days Verifies that the null percent value in a column changes in a rate within a percentile boundary during last 30 days. ColumnAnomalyStationaryNullPercent30DaysCheckSpec
daily_partition_nulls_percent_anomaly_stationary Verifies that the null percent value in a column changes in a rate within a percentile boundary during last 90 days. ColumnAnomalyStationaryNullPercentCheckSpec
daily_partition_nulls_percent_change Verifies that the null percent value in a column changed in a fixed rate since last readout. ColumnChangeNullPercentCheckSpec
daily_partition_nulls_percent_change_yesterday Verifies that the null percent value in a column changed in a fixed rate since last readout from yesterday. ColumnChangeNullPercentSinceYesterdayCheckSpec
daily_partition_nulls_percent_change_7_days Verifies that the null percent value in a column changed in a fixed rate since last readout from last week. ColumnChangeNullPercentSince7DaysCheckSpec
daily_partition_nulls_percent_change_30_days Verifies that the null percent value in a column changed in a fixed rate since last readout from last month. ColumnChangeNullPercentSince30DaysCheckSpec
daily_partition_not_nulls_count Verifies that the number of not null values in a column does not exceed the set count. Creates a separate data quality check (and an alert) for each daily partition. ColumnNotNullsCountCheckSpec
daily_partition_not_nulls_percent Verifies that the percentage of not null values in a column does not exceed the set percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnNotNullsPercentCheckSpec

ColumnSqlDailyPartitionedChecksSpec

Container of built-in preconfigured data quality checks on a column level that are using custom SQL expressions (conditions).

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_sql_condition_passed_percent_on_column Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Creates a separate data quality check (and an alert) for each daily partition. ColumnSqlConditionPassedPercentCheckSpec
daily_partition_sql_condition_failed_count_on_column Verifies that a number of rows failed a custom SQL condition(expression) does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnSqlConditionFailedCountCheckSpec
daily_partition_sql_aggregate_expr_column Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnSqlAggregateExprCheckSpec

ColumnIntegrityDailyPartitionedChecksSpec

Container of integrity data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_foreign_key_not_match_count Verifies that the number of values in a column that does not match values in another table column does not exceed the set count. Creates a separate data quality check (and an alert) for each daily partition. ColumnIntegrityForeignKeyNotMatchCountCheckSpec
daily_partition_foreign_key_match_percent Verifies that the percentage of values in a column that matches values in another table column does not exceed the set count. Creates a separate data quality check (and an alert) for each daily partition. ColumnIntegrityForeignKeyMatchPercentCheckSpec

ColumnDatatypeDailyPartitionedChecksSpec

Container of datatype data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_date_match_format_percent Verifies that the percentage of date values matching the given format in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnDatatypeDateMatchFormatPercentCheckSpec
daily_partition_string_datatype_changed Detects that the data type of texts stored in a text column has changed when compared to an earlier not empty partition. The sensor returns the detected data type of a column: 1 - integers, 2 - floats, 3 - dates, 4 - timestamps, 5 - booleans, 6 - strings, 7 - mixed data types. Creates a separate data quality check (and an alert) for each daily partition. ColumnDatatypeStringDatatypeChangedCheckSpec

ColumnBoolDailyPartitionedChecksSpec

Container of boolean data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_true_percent Verifies that the percentage of true values in a column does not exceed the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnTruePercentCheckSpec
daily_partition_false_percent Verifies that the percentage of false values in a column does not exceed the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnFalsePercentCheckSpec

ColumnStringsDailyPartitionedChecksSpec

Container of strings data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_string_max_length Verifies that the length of string in a column does not exceed the maximum accepted length. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringMaxLengthCheckSpec
daily_partition_string_min_length Verifies that the length of string in a column does not fall below the minimum accepted length. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringMinLengthCheckSpec
daily_partition_string_mean_length Verifies that the length of string in a column does not exceed the mean accepted length. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringMeanLengthCheckSpec
daily_partition_string_length_below_min_length_count The check counts the number of strings in the column that is below the length defined by the user as a parameter. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringLengthBelowMinLengthCountCheckSpec
daily_partition_string_length_below_min_length_percent The check counts the percentage of strings in the column that is below the length defined by the user as a parameter. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringLengthBelowMinLengthPercentCheckSpec
daily_partition_string_length_above_max_length_count The check counts the number of strings in the column that is above the length defined by the user as a parameter. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringLengthAboveMaxLengthCountCheckSpec
daily_partition_string_length_above_max_length_percent The check counts the percentage of strings in the column that is above the length defined by the user as a parameter. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringLengthAboveMaxLengthPercentCheckSpec
daily_partition_string_length_in_range_percent The check counts the percentage of those strings with length in the range provided by the user in the column. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringLengthInRangePercentCheckSpec
daily_partition_string_empty_count Verifies that the number of empty strings in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringEmptyCountCheckSpec
daily_partition_string_empty_percent Verifies that the percentage of empty strings in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringEmptyPercentCheckSpec
daily_partition_string_whitespace_count Verifies that the number of whitespace strings in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringWhitespaceCountCheckSpec
daily_partition_string_whitespace_percent Verifies that the percentage of whitespace strings in a column does not exceed the maximum accepted percent. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringWhitespacePercentCheckSpec
daily_partition_string_surrounded_by_whitespace_count Verifies that the number of strings surrounded by whitespace in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringSurroundedByWhitespaceCountCheckSpec
daily_partition_string_surrounded_by_whitespace_percent Verifies that the percentage of strings surrounded by whitespace in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringSurroundedByWhitespacePercentCheckSpec
daily_partition_string_null_placeholder_count Verifies that the number of null placeholders in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringNullPlaceholderCountCheckSpec
daily_partition_string_null_placeholder_percent Verifies that the percentage of null placeholders in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringNullPlaceholderPercentCheckSpec
daily_partition_string_boolean_placeholder_percent Verifies that the percentage of boolean placeholder for strings in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringBooleanPlaceholderPercentCheckSpec
daily_partition_string_parsable_to_integer_percent Verifies that the percentage of parsable to integer string in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringParsableToIntegerPercentCheckSpec
daily_partition_string_parsable_to_float_percent Verifies that the percentage of parsable to float string in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringParsableToFloatPercentCheckSpec
daily_partition_expected_strings_in_use_count Verifies that the expected string values were found in the column. Raises a data quality issue when too many expected values were not found (were missing). Creates a separate data quality check (and an alert) for each daily partition. ColumnExpectedStringsInUseCountCheckSpec
daily_partition_string_value_in_set_percent The check measures the percentage of rows whose value in a tested column is one of values from a list of expected values or the column value is null. Verifies that the percentage of rows having a valid column value does not exceed the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringValueInSetPercentCheckSpec
daily_partition_string_valid_dates_percent Verifies that the percentage of valid dates in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringValidDatesPercentCheckSpec
daily_partition_string_valid_country_code_percent Verifies that the percentage of valid country code in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringValidCountryCodePercentCheckSpec
daily_partition_string_valid_currency_code_percent Verifies that the percentage of valid currency code in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringValidCurrencyCodePercentCheckSpec
daily_partition_string_invalid_email_count Verifies that the number of invalid emails in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringInvalidEmailCountCheckSpec
daily_partition_string_invalid_uuid_count Verifies that the number of invalid UUID in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringInvalidUuidCountCheckSpec
daily_partition_valid_uuid_percent Verifies that the percentage of valid UUID in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringValidUuidPercentCheckSpec
daily_partition_string_invalid_ip4_address_count Verifies that the number of invalid IP4 address in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringInvalidIp4AddressCountCheckSpec
daily_partition_string_invalid_ip6_address_count Verifies that the number of invalid IP6 address in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringInvalidIp6AddressCountCheckSpec
daily_partition_string_not_match_regex_count Verifies that the number of strings not matching the custom regex in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringNotMatchRegexCountCheckSpec
daily_partition_string_match_regex_percent Verifies that the percentage of strings matching the custom regex in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringMatchRegexPercentCheckSpec
daily_partition_string_not_match_date_regex_count Verifies that the number of strings not matching the date format regex in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringNotMatchDateRegexCountCheckSpec
daily_partition_string_match_date_regex_percent Verifies that the percentage of strings matching the date format regex in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringMatchDateRegexPercentCheckSpec
daily_partition_string_match_name_regex_percent Verifies that the percentage of strings matching the name format regex in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringMatchNameRegexPercentCheckSpec
daily_partition_expected_strings_in_top_values_count Verifies that the top X most popular column values contain all values from a list of expected values. Creates a separate data quality check (and an alert) for each daily partition. ColumnExpectedStringsInTopValuesCountCheckSpec
daily_partition_string_datatype_detected Detects the data type of text values stored in the column. The sensor returns the code of the detected data type of a column: 1 - integers, 2 - floats, 3 - dates, 4 - timestamps, 5 - booleans, 6 - strings, 7 - mixed data types. Raises a data quality issue when the detected data type does not match the expected data type. Creates a separate data quality check (and an alert) for each daily partition. ColumnStringDatatypeDetectedCheckSpec

ColumnDatetimeDailyPartitionedChecksSpec

Container of date-time data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_date_values_in_future_percent Verifies that the percentage of date values in future in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnDateValuesInFuturePercentCheckSpec
daily_partition_datetime_value_in_range_date_percent Verifies that the percentage of date values in the range defined by the user in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnDatetimeValueInRangeDatePercentCheckSpec

ColumnComparisonDailyPartitionedChecksSpec

Container of built-in preconfigured column level comparison checks that compare min/max/sum/mean/nulls measures between the column in the tested (parent) table and a matching reference column in the reference table (the source of truth). This is the configuration for daily partitioned checks that are counted in KPIs.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_sum_match Verifies that percentage of the difference between the sum of values in a tested column in a parent table and the sum of a values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). ColumnComparisonSumMatchCheckSpec
daily_partition_min_match Verifies that percentage of the difference between the minimum value in a tested column in a parent table and the minimum value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). ColumnComparisonMinMatchCheckSpec
daily_partition_max_match Verifies that percentage of the difference between the maximum value in a tested column in a parent table and the maximum value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). ColumnComparisonMaxMatchCheckSpec
daily_partition_mean_match Verifies that percentage of the difference between the mean (average) value in a tested column in a parent table and the mean (average) value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). ColumnComparisonMeanMatchCheckSpec
daily_partition_not_null_count_match Verifies that percentage of the difference between the count of not null values in a tested column in a parent table and the count of not null values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). ColumnComparisonNotNullCountMatchCheckSpec
daily_partition_null_count_match Verifies that percentage of the difference between the count of null values in a tested column in a parent table and the count of null values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). ColumnComparisonNullCountMatchCheckSpec
reference_column The name of the reference column name in the reference table. It is the column to which the current column is compared to. string

ColumnComparisonDailyPartitionedChecksSpecMap

Container of comparison checks for each defined data comparison. The name of the key in this dictionary must match a name of a table comparison that is defined on the parent table. Contains configuration of column level comparison checks. Each column level check container also defines the name of the reference column name to which we are comparing.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
access_order boolean
size integer
mod_count integer
threshold integer

ColumnAccuracyDailyPartitionedChecksSpec

Container of accuracy data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 

ColumnPiiDailyPartitionedChecksSpec

Container of PII data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_valid_usa_phone_percent Verifies that the percentage of valid USA phone values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiValidUsaPhonePercentCheckSpec
daily_partition_contains_usa_phone_percent Verifies that the percentage of rows that contains USA phone number in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiContainsUsaPhonePercentCheckSpec
daily_partition_valid_usa_zipcode_percent Verifies that the percentage of valid USA zip code values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiValidUsaZipcodePercentCheckSpec
daily_partition_contains_usa_zipcode_percent Verifies that the percentage of rows that contains USA zip code in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiContainsUsaZipcodePercentCheckSpec
daily_partition_valid_email_percent Verifies that the percentage of valid emails values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiValidEmailPercentCheckSpec
daily_partition_contains_email_percent Verifies that the percentage of rows that contains emails in a column does not exceed the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiContainsEmailPercentCheckSpec
daily_partition_valid_ip4_address_percent Verifies that the percentage of valid IP4 address values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiValidIp4AddressPercentCheckSpec
daily_partition_contains_ip4_percent Verifies that the percentage of rows that contains IP4 address values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiContainsIp4PercentCheckSpec
daily_partition_valid_ip6_address_percent Verifies that the percentage of valid IP6 address values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiValidIp6AddressPercentCheckSpec
daily_partition_contains_ip6_percent Verifies that the percentage of rows that contains valid IP6 address values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnPiiContainsIp6PercentCheckSpec

ColumnAnomalyDailyPartitionedChecksSpec

Container of built-in preconfigured data quality checks on a column level for detecting anomalies.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_mean_anomaly_stationary_30_days Verifies that the mean value in a column is within a percentile from measurements made during the last 30 days. ColumnAnomalyStationaryMean30DaysCheckSpec
daily_partition_mean_anomaly_stationary Verifies that the mean value in a column is within a percentile from measurements made during the last 90 days. ColumnAnomalyStationaryMeanCheckSpec
daily_partition_median_anomaly_stationary_30_days Verifies that the median in a column is within a percentile from measurements made during the last 30 days. ColumnAnomalyStationaryMedian30DaysCheckSpec
daily_partition_median_anomaly_stationary Verifies that the median in a column is within a percentile from measurements made during the last 90 days. ColumnAnomalyStationaryMedianCheckSpec
daily_partition_sum_anomaly_stationary_30_days Verifies that the sum in a column is within a percentile from measurements made during the last 30 days. ColumnAnomalyStationaryPartitionSum30DaysCheckSpec
daily_partition_sum_anomaly_stationary Verifies that the sum in a column is within a percentile from measurements made during the last 90 days. ColumnAnomalyStationaryPartitionSumCheckSpec
daily_partition_mean_change Verifies that the mean value in a column changed in a fixed rate since last readout. ColumnChangeMeanCheckSpec
daily_partition_mean_change_yesterday Verifies that the mean value in a column changed in a fixed rate since last readout from yesterday. ColumnChangeMeanSinceYesterdayCheckSpec
daily_partition_mean_change_7_days Verifies that the mean value in a column changed in a fixed rate since last readout from last week. ColumnChangeMeanSince7DaysCheckSpec
daily_partition_mean_change_30_days Verifies that the mean value in a column changed in a fixed rate since last readout from last month. ColumnChangeMeanSince30DaysCheckSpec
daily_partition_median_change Verifies that the median in a column changed in a fixed rate since last readout. ColumnChangeMedianCheckSpec
daily_partition_median_change_yesterday Verifies that the median in a column changed in a fixed rate since last readout from yesterday. ColumnChangeMedianSinceYesterdayCheckSpec
daily_partition_median_change_7_days Verifies that the median in a column changed in a fixed rate since last readout from last week. ColumnChangeMedianSince7DaysCheckSpec
daily_partition_median_change_30_days Verifies that the median in a column changed in a fixed rate since last readout from last month. ColumnChangeMedianSince30DaysCheckSpec
daily_partition_sum_change Verifies that the sum in a column changed in a fixed rate since last readout. ColumnChangeSumCheckSpec
daily_partition_sum_change_yesterday Verifies that the sum in a column changed in a fixed rate since last readout from yesterday. ColumnChangeSumSinceYesterdayCheckSpec
daily_partition_sum_change_7_days Verifies that the sum in a column changed in a fixed rate since last readout from last week. ColumnChangeSumSince7DaysCheckSpec
daily_partition_sum_change_30_days Verifies that the sum in a column changed in a fixed rate since last readout from last month. ColumnChangeSumSince30DaysCheckSpec

ColumnNumericDailyPartitionedChecksSpec

Container of numeric data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_negative_count Verifies that the number of negative values in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnNegativeCountCheckSpec
daily_partition_negative_percent Verifies that the percentage of negative values in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnNegativePercentCheckSpec
daily_partition_non_negative_count Verifies that the number of non-negative values in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnNonNegativeCountCheckSpec
daily_partition_non_negative_percent Verifies that the percentage of non-negative values in a column does not exceed the maximum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnNonNegativePercentCheckSpec
daily_partition_expected_numbers_in_use_count Verifies that the expected numeric values were found in the column. Raises a data quality issue when too many expected values were not found (were missing). Creates a separate data quality check (and an alert) for each daily partition. ColumnExpectedNumbersInUseCountCheckSpec
daily_partition_number_value_in_set_percent The check measures the percentage of rows whose value in a tested column is one of values from a list of expected values or the column value is null. Verifies that the percentage of rows having a valid column value does not exceed the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnNumberValueInSetPercentCheckSpec
daily_partition_values_in_range_numeric_percent Verifies that the percentage of values from range in a column does not exceed the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnValuesInRangeNumericPercentCheckSpec
daily_partition_values_in_range_integers_percent Verifies that the percentage of values from range in a column does not exceed the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnValuesInRangeIntegersPercentCheckSpec
daily_partition_value_below_min_value_count The check counts the number of values in the column that is below the value defined by the user as a parameter. Creates a separate data quality check (and an alert) for each daily partition. ColumnValueBelowMinValueCountCheckSpec
daily_partition_value_below_min_value_percent The check counts the percentage of values in the column that is below the value defined by the user as a parameter. Creates a separate data quality check (and an alert) for each daily partition. ColumnValueBelowMinValuePercentCheckSpec
daily_partition_value_above_max_value_count The check counts the number of values in the column that is above the value defined by the user as a parameter. Creates a separate data quality check (and an alert) for each daily partition. ColumnValueAboveMaxValueCountCheckSpec
daily_partition_value_above_max_value_percent The check counts the percentage of values in the column that is above the value defined by the user as a parameter. Creates a separate data quality check (and an alert) for each daily partition. ColumnValueAboveMaxValuePercentCheckSpec
daily_partition_max_in_range Verifies that the maximal value in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnMaxInRangeCheckSpec
daily_partition_min_in_range Verifies that the minimal value in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnMinInRangeCheckSpec
daily_partition_mean_in_range Verifies that the average (mean) of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnMeanInRangeCheckSpec
daily_partition_percentile_in_range Verifies that the percentile of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnPercentileInRangeCheckSpec
daily_partition_median_in_range Verifies that the median of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnMedianInRangeCheckSpec
daily_partition_percentile_10_in_range Verifies that the percentile 10 of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnPercentile10InRangeCheckSpec
daily_partition_percentile_25_in_range Verifies that the percentile 25 of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnPercentile25InRangeCheckSpec
daily_partition_percentile_75_in_range Verifies that the percentile 75 of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnPercentile75InRangeCheckSpec
daily_partition_percentile_90_in_range Verifies that the percentile 90 of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnPercentile90InRangeCheckSpec
daily_partition_sample_stddev_in_range Verifies that the sample standard deviation of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnSampleStddevInRangeCheckSpec
daily_partition_population_stddev_in_range Verifies that the population standard deviation of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnPopulationStddevInRangeCheckSpec
daily_partition_sample_variance_in_range Verifies that the sample variance of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnSampleVarianceInRangeCheckSpec
daily_partition_population_variance_in_range Verifies that the population variance of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnPopulationVarianceInRangeCheckSpec
daily_partition_sum_in_range Verifies that the sum of all values in a column is not outside the set range. Creates a separate data quality check (and an alert) for each daily partition. ColumnSumInRangeCheckSpec
daily_partition_invalid_latitude_count Verifies that the number of invalid latitude values in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnInvalidLatitudeCountCheckSpec
daily_partition_valid_latitude_percent Verifies that the percentage of valid latitude values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnValidLatitudePercentCheckSpec
daily_partition_invalid_longitude_count Verifies that the number of invalid longitude values in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnInvalidLongitudeCountCheckSpec
daily_partition_valid_longitude_percent Verifies that the percentage of valid longitude values in a column does not fall below the minimum accepted percentage. Creates a separate data quality check (and an alert) for each daily partition. ColumnValidLongitudePercentCheckSpec

ColumnDailyPartitionedCheckCategoriesSpec

Container of data quality partitioned checks on a column level that are checking numeric values at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
nulls Daily partitioned checks of nulls in the column ColumnNullsDailyPartitionedChecksSpec
numeric Daily partitioned checks of numeric in the column ColumnNumericDailyPartitionedChecksSpec
strings Daily partitioned checks of strings in the column ColumnStringsDailyPartitionedChecksSpec
uniqueness Daily partitioned checks of uniqueness in the column ColumnUniquenessDailyPartitionedChecksSpec
datetime Daily partitioned checks of datetime in the column ColumnDatetimeDailyPartitionedChecksSpec
pii Daily partitioned checks of Personal Identifiable Information (PII) in the column ColumnPiiDailyPartitionedChecksSpec
sql Daily partitioned checks using custom SQL expressions evaluated on the column ColumnSqlDailyPartitionedChecksSpec
bool Daily partitioned checks for booleans in the column ColumnBoolDailyPartitionedChecksSpec
integrity Daily partitioned checks for integrity in the column ColumnIntegrityDailyPartitionedChecksSpec
accuracy Daily partitioned checks for accuracy in the column ColumnAccuracyDailyPartitionedChecksSpec
datatype Daily partitioned checks for datatype in the column ColumnDatatypeDailyPartitionedChecksSpec
anomaly Daily partitioned checks for anomaly in the column ColumnAnomalyDailyPartitionedChecksSpec
comparisons Dictionary of configuration of checks for table comparisons at a column level. The key that identifies each comparison must match the name of a data comparison that is configured on the parent table. ColumnComparisonDailyPartitionedChecksSpecMap
custom Dictionary of custom checks. The keys are check names. CustomCheckSpecMap

ColumnUniquenessDailyPartitionedChecksSpec

Container of uniqueness data quality partitioned checks on a column level that are checking at a daily level.

The structure of this object is described below

 Property name   Description                       Data type   Enum values   Default value   Sample values 
daily_partition_distinct_count Verifies that the number of distinct values in a column does not fall below the minimum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnDistinctCountCheckSpec
daily_partition_distinct_percent Verifies that the percentage of distinct values in a column does not fall below the minimum accepted percent. Creates a separate data quality check (and an alert) for each daily partition. ColumnDistinctPercentCheckSpec
daily_partition_duplicate_count Verifies that the number of duplicate values in a column does not exceed the maximum accepted count. Creates a separate data quality check (and an alert) for each daily partition. ColumnDuplicateCountCheckSpec
daily_partition_duplicate_percent Verifies that the percent of duplicate values in a column does not exceed the maximum accepted percent. Creates a separate data quality check (and an alert) for each daily partition. ColumnDuplicatePercentCheckSpec