Last updated: July 22, 2025

List of column level custom sql data quality checks

This is a list of custom_sql column data quality checks supported by DQOps and a brief description of what data quality issued they detect.

column-level custom_sql checks

Validate data against user-defined SQL queries at the column level. Checks in this group allow to validate whether a set percentage of rows has passed a custom SQL expression or whether the custom SQL expression is not outside the set range.

sql condition failed on column

A column-level check that uses a custom SQL expression on each column to verify (assert) that all rows pass a custom condition defined as an SQL expression. Use the {alias} token to reference the tested table, and the {column} to reference the column that is tested. This data quality check can be used to compare columns on the same table. For example, when this check is applied on a col_price column, the condition can verify that the col_price is higher than the col_tax using an SQL expression: `{alias}.{column} > {alias}.col_tax` Use an SQL expression that returns a true value for valid values and false for invalid values, because it is an assertion.

Data quality check name	Friendly name	Check type	Description
`profile_sql_condition_failed_on_column`	Maximum count of rows that failed SQL conditions	profiling	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between the current column and another column: `{alias}.{column} > col_tax`.
`daily_sql_condition_failed_on_column`	Maximum count of rows that failed SQL conditions	monitoring	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between the current column and another column: `{alias}.{column} > col_tax`. Stores the most recent captured count of failed rows for each day when the data quality check was evaluated.
`monthly_sql_condition_failed_on_column`	Maximum count of rows that failed SQL conditions	monitoring	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between the current column and another column: `{alias}.{column} > {alias}.col_tax`. Stores the most recent captured count of failed rows for each month when the data quality check was evaluated.
`daily_partition_sql_condition_failed_on_column`	Maximum count of rows that failed SQL conditions	partitioned	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between the current column and another column: `{alias}.{column} > {alias}.col_tax`. Stores a separate data quality check result for each daily partition.
`monthly_partition_sql_condition_failed_on_column`	Maximum count of rows that failed SQL conditions	partitioned	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between the current column and another column: `{alias}.{column} > {alias}.col_tax`. Stores a separate data quality check result for each monthly partition.

sql condition passed percent on column

A table-level check that ensures that a minimum percentage of rows passed a custom SQL condition (expression). Measures the percentage of rows passing the condition. Raises a data quality issue when the percent of valid rows is below the min_percent parameter.

Data quality check name	Friendly name	Check type	Description
`profile_sql_condition_passed_percent_on_column`	Minimum percentage of rows that passed SQL condition	profiling	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current column by using tokens, for example: `{alias}.{column} > {alias}.col_tax`.
`daily_sql_condition_passed_percent_on_column`	Minimum percentage of rows that passed SQL condition	monitoring	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current column by using tokens, for example: `{alias}.{column} > {alias}.col_tax`. Stores the most recent captured value for each day when the data quality check was evaluated.
`monthly_sql_condition_passed_percent_on_column`	Minimum percentage of rows that passed SQL condition	monitoring	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current column by using tokens, for example: `{alias}.{column} > {alias}.col_tax`. Stores the most recent check result for each month when the data quality check was evaluated.
`daily_partition_sql_condition_passed_percent_on_column`	Minimum percentage of rows that passed SQL condition	partitioned	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current column by using tokens, for example: `{alias}.{column} > {alias}.col_tax`. Stores a separate data quality check result for each daily partition.
`monthly_partition_sql_condition_passed_percent_on_column`	Minimum percentage of rows that passed SQL condition	partitioned	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current column by using tokens, for example: `{alias}.{column} > {alias}.col_tax`. Stores a separate data quality check result for each monthly partition.

sql aggregate expression on column

A column-level check that calculates a given SQL aggregate expression on a column and verifies if the value is within a range of accepted values.

Data quality check name	Friendly name	Check type	Description
`profile_sql_aggregate_expression_on_column`	Custom aggregated SQL expression within range	profiling	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range.
`daily_sql_aggregate_expression_on_column`	Custom aggregated SQL expression within range	monitoring	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range. Stores the most recent captured value for each day when the data quality check was evaluated.
`monthly_sql_aggregate_expression_on_column`	Custom aggregated SQL expression within range	monitoring	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range. Stores the most recent check result for each month when the data quality check was evaluated.
`daily_partition_sql_aggregate_expression_on_column`	Custom aggregated SQL expression within range	partitioned	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range. Stores a separate data quality check result for each daily partition.
`monthly_partition_sql_aggregate_expression_on_column`	Custom aggregated SQL expression within range	partitioned	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range. Stores a separate data quality check result for each monthly partition.

sql invalid value count on column

A column-level check that uses a custom SQL query that return invalid values from column. This check is used for setting testing queries or ready queries used by users in their own systems (legacy SQL queries). Use the {table} token to reference the tested table, and the {column} to reference the column that is tested. For example, when this check is applied on a column. The condition can find invalid values in the column which have values lower than 18 using an SQL query: `SELECT {column} FROM {table} WHERE {column} < 18`.

Data quality check name	Friendly name	Check type	Description
`profile_sql_invalid_value_count_on_column`	Custom SELECT SQL that returns invalid values	profiling	Runs a custom query that retrieves invalid values found in a column and returns the number of them, and raises an issue if too many failures were detected. This check is used for setting testing queries or ready queries used by users in their own systems (legacy SQL queries). For example, when this check is applied on a column. The condition can find invalid values in the column which have values lower than 18 using an SQL query: `SELECT {column} FROM {table} WHERE {column} < 18`.
`daily_sql_invalid_value_count_on_column`	Custom SELECT SQL that returns invalid values	monitoring	Runs a custom query that retrieves invalid values found in a column and returns the number of them, and raises an issue if too many failures were detected. This check is used for setting testing queries or ready queries used by users in their own systems (legacy SQL queries). For example, when this check is applied on a column. The condition can find invalid values in the column which have values lower than 18 using an SQL query: `SELECT {column} FROM {table} WHERE {column} < 18`.
`monthly_sql_invalid_value_count_on_column`	Custom SELECT SQL that returns invalid values	monitoring	Runs a custom query that retrieves invalid values found in a column and returns the number of them, and raises an issue if too many failures were detected. This check is used for setting testing queries or ready queries used by users in their own systems (legacy SQL queries). For example, when this check is applied on a column. The condition can find invalid values in the column which have values lower than 18 using an SQL query: `SELECT {column} FROM {table} WHERE {column} < 18`.

import custom result on column

Column level check that uses a custom SQL SELECT statement to retrieve a result of running a custom data quality check on a column by a custom data quality check, hardcoded in the data pipeline. The result is retrieved by querying a separate logging table, whose schema is not fixed. The logging table should have columns that identify a table and a column for which they store custom data quality check results, and a severity column of the data quality issue. The SQL query that is configured in this external data quality results importer must be a complete SELECT statement that queries a dedicated logging table, created by the data engineering team.

Data quality check name	Friendly name	Check type	Description
`profile_import_custom_result_on_column`	Import custom data quality results on column	profiling	Runs a custom query that retrieves a result of a data quality check performed in the data engineering, whose result (the severity level) is pulled from a separate table.
`daily_import_custom_result_on_column`	Import custom data quality results on column	monitoring	Runs a custom query that retrieves a result of a data quality check performed in the data engineering, whose result (the severity level) is pulled from a separate table.
`monthly_import_custom_result_on_column`	Import custom data quality results on column	monitoring	Runs a custom query that retrieves a result of a data quality check performed in the data engineering, whose result (the severity level) is pulled from a separate table.