Last updated: July 22, 2025

List of table level custom sql data quality checks

This is a list of custom_sql table data quality checks supported by DQOps and a brief description of what data quality issued they detect.

table-level custom_sql checks

Validate data against user-defined SQL queries at the table level. Checks in this group allow for validation that the set percentage of rows passed a custom SQL expression or that the custom SQL expression is not outside the set range.

sql condition failed on table

A table-level check that uses a custom SQL expression on each row to verify (assert) that all rows pass a custom condition defined as an SQL condition. Use the {alias} token to reference the tested table. This data quality check can be used to compare columns on the same table. For example, the condition can verify that the value in the col_price column is higher than the col_tax column using an SQL expression: `{alias}.col_price > {alias}.col_tax`. Use an SQL expression that returns a true value for valid values and a false one for invalid values, because it is an assertion.

Data quality check name	Friendly name	Check type	Description
`profile_sql_condition_failed_on_table`	Maximum count of rows that failed SQL conditions	profiling	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current table by using tokens, for example: `{alias}.col_price > {alias}.col_tax`.
`daily_sql_condition_failed_on_table`	Maximum count of rows that failed SQL conditions	monitoring	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between columns: `{alias}.col_price > {alias}.col_tax`. Stores the most recent count of failed rows for each day when the data quality check was evaluated.
`monthly_sql_condition_failed_on_table`	Maximum count of rows that failed SQL conditions	monitoring	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between columns: `{alias}.col_price > {alias}.col_tax`. Stores the most recent count of failed rows for each month when the data quality check was evaluated.
`daily_partition_sql_condition_failed_on_table`	Maximum count of rows that failed SQL conditions	partitioned	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between columns: `{alias}.col_price > {alias}.col_tax`. Stores a separate data quality check result for each daily partition.
`monthly_partition_sql_condition_failed_on_table`	Maximum count of rows that failed SQL conditions	partitioned	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between columns: `{alias}.col_price > {alias}.col_tax`. Stores a separate data quality check result for each monthly partition.

sql condition passed percent on table

A table-level check that ensures that a minimum percentage of rows passed a custom SQL condition (expression). Measures the percentage of rows passing the condition. Raises a data quality issue when the percent of valid rows is below the min_percent parameter.

Data quality check name	Friendly name	Check type	Description
`profile_sql_condition_passed_percent_on_table`	Minimum percentage of rows that passed SQL condition	profiling	Verifies that a custom SQL expression is met for each row. Counts the number of rows where the expression is not satisfied, and raises an issue if too many failures were detected. This check is used also to compare values between columns: `{alias}.col_price > {alias}.col_tax`.
`daily_sql_condition_passed_percent_on_table`	Minimum percentage of rows that passed SQL condition	monitoring	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current table by using tokens, for example: `{alias}.col_price > {alias}.col_tax`. Stores the most recent captured percentage for each day when the data quality check was evaluated.
`monthly_sql_condition_passed_percent_on_table`	Minimum percentage of rows that passed SQL condition	monitoring	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current table by using tokens, for example: `{alias}.col_price > {alias}.col_tax`. Stores the most recent value for each month when the data quality check was evaluated.
`daily_partition_sql_condition_passed_percent_on_table`	Minimum percentage of rows that passed SQL condition	partitioned	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current table by using tokens, for example: `{alias}.col_price > {alias}.col_tax`. Stores a separate data quality check result for each daily partition.
`monthly_partition_sql_condition_passed_percent_on_table`	Minimum percentage of rows that passed SQL condition	partitioned	Verifies that a minimum percentage of rows passed a custom SQL condition (expression). Reference the current table by using tokens, for example: `{alias}.col_price > {alias}.col_tax`. Stores a separate data quality check result for each monthly partition.

sql aggregate expression on table

A table-level check that calculates a given SQL aggregate expression on a table and verifies if the value is within a range of accepted values.

Data quality check name	Friendly name	Check type	Description
`profile_sql_aggregate_expression_on_table`	Custom aggregated SQL expression within range	profiling	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range.
`daily_sql_aggregate_expression_on_table`	Custom aggregated SQL expression within range	monitoring	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range. Stores the most recent captured value for each day when the data quality check was evaluated.
`monthly_sql_aggregate_expression_on_table`	Custom aggregated SQL expression within range	monitoring	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range. Stores the most recent value for each month when the data quality check was evaluated.
`daily_partition_sql_aggregate_expression_on_table`	Custom aggregated SQL expression within range	partitioned	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range. Stores a separate data quality check result for each daily partition.
`monthly_partition_sql_aggregate_expression_on_table`	Custom aggregated SQL expression within range	partitioned	Verifies that a custom aggregated SQL expression (MIN, MAX, etc.) is not outside the expected range. Stores a separate data quality check result for each monthly partition.

sql invalid record count on table

A table-level check that uses a custom SQL query that return invalid values from column. Use the {table} token to reference the tested table. This data quality check can be used to compare columns on the same table. For example, when this check is applied on a age column, the condition can find invalid records in which the age is lower than 18 using an SQL query: `SELECT age FROM {table} WHERE age < 18`.

Data quality check name	Friendly name	Check type	Description
`profile_sql_invalid_record_count_on_table`	Custom SELECT SQL that returns invalid records	profiling	Runs a custom query that retrieves invalid records found in a table and returns the number of them, and raises an issue if too many failures were detected. This check is used for setting testing queries or ready queries used by users in their own systems (legacy SQL queries). For example, when this check is applied on a age column, the condition can find invalid records in which the age is lower than 18 using an SQL query: `SELECT age FROM {table} WHERE age < 18`.
`daily_sql_invalid_record_count_on_table`	Custom SELECT SQL that returns invalid records	monitoring	Runs a custom query that retrieves invalid records found in a table and returns the number of them, and raises an issue if too many failures were detected. This check is used for setting testing queries or ready queries used by users in their own systems (legacy SQL queries). For example, when this check is applied on a age column, the condition can find invalid records in which the age is lower than 18 using an SQL query: `SELECT age FROM {table} WHERE age < 18`.
`monthly_sql_invalid_record_count_on_table`	Custom SELECT SQL that returns invalid records	monitoring	Runs a custom query that retrieves invalid records found in a table and returns the number of them, and raises an issue if too many failures were detected. This check is used for setting testing queries or ready queries used by users in their own systems (legacy SQL queries). For example, when this check is applied on a age column, the condition can find invalid records in which the age is lower than 18 using an SQL query: `SELECT age FROM {table} WHERE age < 18`.

import custom result on table

A table-level check that uses a custom SQL SELECT statement to retrieve a result of running a custom data quality check that was hardcoded in the data pipeline, and the result was stored in a separate table. The SQL query that is configured in this external data quality results importer must be a complete SELECT statement that queries a dedicated table (created by the data engineers) that stores the results of custom data quality checks. The SQL query must return a severity column with values: 0 - data quality check passed, 1 - warning issue, 2 - error severity issue, 3 - fatal severity issue.

Data quality check name	Friendly name	Check type	Description
`profile_import_custom_result_on_table`	Import custom data quality results on table	profiling	Runs a custom query that retrieves a result of a data quality check performed in the data engineering, whose result (the severity level) is pulled from a separate table.
`daily_import_custom_result_on_table`	Import custom data quality results on table	monitoring	Runs a custom query that retrieves a result of a data quality check performed in the data engineering, whose result (the severity level) is pulled from a separate table.
`monthly_import_custom_result_on_table`	Import custom data quality results on table	monitoring	Runs a custom query that retrieves a result of a data quality check performed in the data engineering, whose result (the severity level) is pulled from a separate table.