Data Quality Operations Center
Profile, automate, and monitor data quality
DQOps is an open-source data quality platform for all stages, from profiling new data sources to automating data quality and detecting issues with Data Observability
End-to-End Data Quality Platform
Begin with self-service data profiling, automate data quality checks from data pipelines,
and let Data Observability detect schema changes, anomalies,
and invalid data in your data platforms
Self-service data profiling
Profile data sources and validate data with 150+ data quality checks using a local DQOps instance or a shared multi-user environment.
Integrate into data pipelines
Run data quality checks from data pipelines, verify data contracts of source tables, and prevent corrupted data from being loaded.
Detect and manage issues
Identify data anomalies and schema drifts.
Group issues into data quality incidents and assign incidents to the right team.
Measure data quality KPI scores
Measure data quality with KPIs that you can prove to business sponsors. Verify data quality SLAs for data domains and vendors.
How it works
Connect your data sources, start monitoring data, integrate data quality checks into data pipelines, and measure data quality with a data quality KPI score.
Data Contracts
Data Contracts
Configure data quality checks in YAML files with full code completion and in-place documentation in Visual Studio Code.
Validate data quality for source and target tables:
- incomplete tables and columns,
- values outside of an accepted list of valid values,
- values not matching patterns,
- apply data quality checks automatically using policies.
Advanced Data Profiling
Start with a simple statistical analysis to get a quick insight into data. Then, run advanced profiling to choose the right data checks to monitor.
- More than 150 built-in table and column data quality checks
- Measure completeness, timeliness, validity, consistency, reasonableness, and accuracy
- Create custom data quality checks and rules with Jinja2 and Python
Start with a simple statistical analysis to get a quick insight into data. Then, run advanced profiling to choose the right data checks to monitor.
- More than 150 built-in table and column data quality checks
- Measure completeness, timeliness, validity, consistency, reasonableness, and accuracy
- Create custom data quality checks and rules with Jinja2 and Python
Anomaly Detection
Anomaly Detection
Automatically observe your data to detect potential data issues as soon as they appear and before anyone else is impacted.
Apply data observability to detect changes in:
- data volume,
- data characteristics min, max, mean and sum,
- missing or not fresh data,
- schema drifts.
Why DQOps
Custom Data Quality Dashboards
Create custom data quality dashboards using our data quality data warehouse.
Analyze
Partitioned
Data
Monitor tables at any scale using incremental data quality monitoring at a partition level.
Custom Data Quality
Rules
Define a custom inventory of approved data quality checks, shared by the data quality team.
DataOps
Friendly
Define data quality definitions in YAML files stored in Git, and run checks from your data pipelines.
Measure Data Quality KPIs
Aggregate all data quality metrics in a dedicated data quality data warehouse. Calculate data quality KPIs as a percentage of passed data quality checks. Use 50+ data quality dashboards.
- Governance, operational, and detailed dashboards
- Build dashboards with a custom Looker Studio Connector
- Prove the quality of the data with numerical KPIs
Aggregate all data quality metrics in a dedicated data quality data warehouse. Calculate data quality KPIs as a percentage of passed data quality checks. Use 50+ data quality dashboards.
- Governance, operational, and detailed dashboards
- Build dashboards with a custom Looker Studio Connector
- Prove the quality of the data with numerical KPIs
Manage Incident Workflows
Manage Incident Workflows
Keep track of the issues that arise during data quality monitoring. Automatically group similar data quality issues into data quality incidents.
- View, filter and manage the incidents
- Assign issues to respective teams
- Automate incident notifications
Monitor Various
Data Sources
Run data quality checks as customizable SQL query templates.
Query the results of your existing custom data quality checks and import them into the data quality warehouse to integrate them into the data quality KPI.
Design any data quality check that can detect business-relevant data quality issues.
Monitor Various
Data Sources
Run data quality checks as customizable SQL query templates.
Query the results of your existing custom data quality checks and import them into the data quality warehouse to integrate them into the data quality KPI.
Design any data quality check that can detect business-relevant data quality issues.