Last updated: July 22, 2025
List of column level comparisons data quality checks
This is a list of comparisons column data quality checks supported by DQOps and a brief description of what data quality issued they detect.
column-level comparisons checks
Compares the columns in a table to another column in another table that is in a different data source.
sum match
A column-level check that ensures that compares the sum of the values in the tested column to the sum of values in a reference column from the reference table. Compares the sum of values for each group of data. The data is grouped using a GROUP BY clause and groups are matched between the tested (parent) table and the reference table (the source of truth).
| Data quality check name | Friendly name | Check type | Description | Standard |
|---|---|---|---|---|
profile_sum_match |
Maximum percentage of difference between sums of compared columns | profiling | Verifies that percentage of the difference between the sum of values in a tested column in a parent table and the sum of a values in a column in the reference table. The difference must be below defined percentage thresholds. | |
daily_sum_match |
Maximum percentage of difference between sums of compared columns | monitoring | Verifies that percentage of the difference between the sum of values in a tested column in a parent table and the sum of a values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each day when the data quality check was evaluated. | |
monthly_sum_match |
Maximum percentage of difference between sums of compared columns | monitoring | Verifies that percentage of the difference between the sum of values in a tested column in a parent table and the sum of a values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. | |
daily_partition_sum_match |
Maximum percentage of difference between sums of compared columns | partitioned | Verifies that percentage of the difference between the sum of values in a tested column in a parent table and the sum of a values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). | |
monthly_partition_sum_match |
Maximum percentage of difference between sums of compared columns | partitioned | Verifies that percentage of the difference between the sum of values in a tested column in a parent table and the sum of a values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each monthly partition (each month of data) between the compared table and the reference table (the source of truth). |
min match
A column-level check that ensures that compares the minimum value in the tested column to the minimum value in a reference column from the reference table. Compares the minimum values for each group of data. The data is grouped using a GROUP BY clause and groups are matched between the tested (parent) table and the reference table (the source of truth).
| Data quality check name | Friendly name | Check type | Description | Standard |
|---|---|---|---|---|
profile_min_match |
Maximum percentage of difference between minimum values of compared columns | profiling | Verifies that percentage of the difference between the minimum value in a tested column in a parent table and the minimum value in a column in the reference table. The difference must be below defined percentage thresholds. | |
daily_min_match |
Maximum percentage of difference between minimum values of compared columns | monitoring | Verifies that percentage of the difference between the minimum value in a tested column in a parent table and the minimum value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each day when the data quality check was evaluated. | |
monthly_min_match |
Maximum percentage of difference between minimum values of compared columns | monitoring | Verifies that percentage of the difference between the minimum value in a tested column in a parent table and the minimum value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. | |
daily_partition_min_match |
Maximum percentage of difference between minimum values of compared columns | partitioned | Verifies that percentage of the difference between the minimum value in a tested column in a parent table and the minimum value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). | |
monthly_partition_min_match |
Maximum percentage of difference between minimum values of compared columns | partitioned | Verifies that percentage of the difference between the minimum value in a tested column in a parent table and the minimum value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each monthly partition (each month of data) between the compared table and the reference table (the source of truth). |
max match
A column-level check that ensures that compares the maximum value in the tested column to maximum value in a reference column from the reference table. Compares the maximum values for each group of data. The data is grouped using a GROUP BY clause and groups are matched between the tested (parent) table and the reference table (the source of truth).
| Data quality check name | Friendly name | Check type | Description | Standard |
|---|---|---|---|---|
profile_max_match |
Maximum percentage of difference between maximum values of compared columns | profiling | Verifies that percentage of the difference between the maximum value in a tested column in a parent table and the maximum value in a column in the reference table. The difference must be below defined percentage thresholds. | |
daily_max_match |
Maximum percentage of difference between maximum values of compared columns | monitoring | Verifies that percentage of the difference between the maximum value in a tested column in a parent table and the maximum value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each day when the data quality check was evaluated. | |
monthly_max_match |
Maximum percentage of difference between maximum values of compared columns | monitoring | Verifies that percentage of the difference between the maximum value in a tested column in a parent table and the maximum value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. | |
daily_partition_max_match |
Maximum percentage of difference between maximum values of compared columns | partitioned | Verifies that percentage of the difference between the maximum value in a tested column in a parent table and the maximum value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). | |
monthly_partition_max_match |
Maximum percentage of difference between maximum values of compared columns | partitioned | Verifies that percentage of the difference between the maximum value in a tested column in a parent table and the maximum value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each monthly partition (each month of data) between the compared table and the reference table (the source of truth). |
mean match
A column-level check that ensures that compares the mean (average) of the values in the tested column to the mean (average) of values in a reference column from the reference table. Compares the mean (average) value for each group of data. The data is grouped using a GROUP BY clause and groups are matched between the tested (parent) table and the reference table (the source of truth).
| Data quality check name | Friendly name | Check type | Description | Standard |
|---|---|---|---|---|
profile_mean_match |
Maximum percentage of difference between mean (average) values of compared columns | profiling | Verifies that percentage of the difference between the mean (average) value in a tested column in a parent table and the mean (average) value in a column in the reference table. The difference must be below defined percentage thresholds. | |
daily_mean_match |
Maximum percentage of difference between mean (average) values of compared columns | monitoring | Verifies that percentage of the difference between the mean (average) value in a tested column in a parent table and the mean (average) value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each day when the data quality check was evaluated. | |
monthly_mean_match |
Maximum percentage of difference between mean (average) values of compared columns | monitoring | Verifies that percentage of the difference between the mean (average) value in a tested column in a parent table and the mean (average) value in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. | |
daily_partition_mean_match |
Maximum percentage of difference between mean (average) values of compared columns | partitioned | Verifies that percentage of the difference between the mean (average) value in a tested column in a parent table and the mean (average) value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). | |
monthly_partition_mean_match |
Maximum percentage of difference between mean (average) values of compared columns | partitioned | Verifies that percentage of the difference between the mean (average) value in a tested column in a parent table and the mean (average) value in a column in the reference table. The difference must be below defined percentage thresholds. Compares each monthly partition (each month of data) between the compared table and the reference table (the source of truth). |
not null count match
A column-level check that ensures that compares the count of not null values in the tested column to the count of not null values in a reference column from the reference table. Compares the count of not null values for each group of data. The data is grouped using a GROUP BY clause and groups are matched between the tested (parent) table and the reference table (the source of truth).
| Data quality check name | Friendly name | Check type | Description | Standard |
|---|---|---|---|---|
profile_not_null_count_match |
Maximum percentage of difference between the counts of not null values in compared columns | profiling | Verifies that percentage of the difference between the count of not null values in a tested column in a parent table and the count of not null values in a column in the reference table. The difference must be below defined percentage thresholds. | |
daily_not_null_count_match |
Maximum percentage of difference between the counts of not null values in compared columns | monitoring | Verifies that percentage of the difference between the count of not null values in a tested column in a parent table and the count of not null values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each day when the data quality check was evaluated. | |
monthly_not_null_count_match |
Maximum percentage of difference between the counts of not null values in compared columns | monitoring | Verifies that percentage of the difference between the count of not null values in a tested column in a parent table and the count of not null values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. | |
daily_partition_not_null_count_match |
Maximum percentage of difference between the counts of not null values in compared columns | partitioned | Verifies that percentage of the difference between the count of not null values in a tested column in a parent table and the count of not null values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). | |
monthly_partition_not_null_count_match |
Maximum percentage of difference between the counts of not null values in compared columns | partitioned | Verifies that percentage of the difference between the count of not null values in a tested column in a parent table and the count of not null values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each monthly partition (each month of data) between the compared table and the reference table (the source of truth). |
null count match
A column-level check that ensures that compares the count of null values in the tested column to the count of null values in a reference column from the reference table. Compares the count of null values for each group of data. The data is grouped using a GROUP BY clause and groups are matched between the tested (parent) table and the reference table (the source of truth).
| Data quality check name | Friendly name | Check type | Description | Standard |
|---|---|---|---|---|
profile_null_count_match |
Maximum percentage of difference between the counts of null values in compared columns | profiling | Verifies that percentage of the difference between the count of null values in a tested column in a parent table and the count of null values in a column in the reference table. The difference must be below defined percentage thresholds. | |
daily_null_count_match |
Maximum percentage of difference between the counts of null values in compared columns | monitoring | Verifies that percentage of the difference between the count of null values in a tested column in a parent table and the count of null values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each day when the data quality check was evaluated. | |
monthly_null_count_match |
Maximum percentage of difference between the counts of null values in compared columns | monitoring | Verifies that percentage of the difference between the count of null values in a tested column in a parent table and the count of null values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. | |
daily_partition_null_count_match |
Maximum percentage of difference between the counts of null values in compared columns | partitioned | Verifies that percentage of the difference between the count of null values in a tested column in a parent table and the count of null values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). | |
monthly_partition_null_count_match |
Maximum percentage of difference between the counts of null values in compared columns | partitioned | Verifies that percentage of the difference between the count of null values in a tested column in a parent table and the count of null values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each monthly partition (each month of data) between the compared table and the reference table (the source of truth). |
distinct count match
A column-level check that ensures that compares the count of distinct values in the tested column to the count of distinct values in a reference column from the reference table. Compares the count of distinct values for each group of data. The data is grouped using a GROUP BY clause and groups are matched between the tested (parent) table and the reference table (the source of truth).
| Data quality check name | Friendly name | Check type | Description | Standard |
|---|---|---|---|---|
profile_distinct_count_match |
Maximum percentage of difference between the counts of distinct values in compared columns | profiling | Verifies that percentage of the difference between the count of distinct values in a tested column in a parent table and the count of distinct values in a column in the reference table. The difference must be below defined percentage thresholds. | |
daily_distinct_count_match |
Maximum percentage of difference between the counts of distinct values in compared columns | monitoring | Verifies that percentage of the difference between the count of distinct values in a tested column in a parent table and the count of distinct values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each day when the data quality check was evaluated. | |
monthly_distinct_count_match |
Maximum percentage of difference between the counts of distinct values in compared columns | monitoring | Verifies that percentage of the difference between the count of distinct values in a tested column in a parent table and the count of distinct values in a column in the reference table. The difference must be below defined percentage thresholds. Stores the most recent captured value for each month when the data quality check was evaluated. | |
daily_partition_distinct_count_match |
Maximum percentage of difference between the counts of distinct values in compared columns | partitioned | Verifies that percentage of the difference between the count of distinct values in a tested column in a parent table and the count of distinct values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each daily partition (each day of data) between the compared table and the reference table (the source of truth). | |
monthly_partition_distinct_count_match |
Maximum percentage of difference between the counts of distinct values in compared columns | partitioned | Verifies that percentage of the difference between the count of distinct values in a tested column in a parent table and the count of distinct values in a column in the reference table. The difference must be below defined percentage thresholds. Compares each monthly partition (each month of data) between the compared table and the reference table (the source of truth). |