How do you ensure that the data exchanged by companies meet the criteria defined in the contract?
Shared Data Quality Rules
Shared Data Quality Rules
As a data provider, define data quality rules and ensure that you deliver data that meet the requirements of the contract.
DQO data quality rules are defined as simple, easy-to-read text files. The data provider shares the file with the customer to prove that those rules are monitored. Run the data quality checks by processing the check in the specification file to ensure that the rules are always met.
- Data quality rules are defined in a simple way and can be shared with the consumer
- Data consumers and data providers can check the same set of rules, even if they store the data on different database platforms
- Data quality rules are ensured and monitored
External Data Verification
External Data Verification
As a consumer of data, verify that the data received from an external party meets the quality requirements.
Define data quality checks for received data and monitor data quality using the DQO data monitoring platform. Data format changes in manually generated files can be detected with multiple quality checks, such as coverage and consistency checks.
- Verify the quality of received data
- Detect missing days or partitions
- Provide valid feedback to the data provider (your external vendor) on which quality rules were violated and which data is missing
Data received on time
Data received on time
Track the timeliness of the data. Check that the data was received within an accepted delay or according to an agreed data delivery schedule.
DQO can detect both the lag of the data and how fresh the data is (the date of the latest rows). DQO can also analyze timeliness over time to detect that longer delays happen more frequently.
- Check the freshness of data (the lag of data delivery)
- Create custom timeliness checks that can compare the delay to a data delivery schedule, for example, the data is received every second Thursday of a month
- Monitor and detect inconsistencies in the data delay by comparing the data delay to the average delay
Real world accuracy checks
Real world accuracy checks
Compare the data received from an external party with other verified data sources.
Accuracy data quality checks can compare data summaries between data received from a data provider and another source, such as the finance department. Accuracy checks are performed as Ground Truth Checks, which compare aggregated summaries grouped by business-relevant dimensions (country, state, etc.).
- Compare the data from an external party with your own data
- Compare your data with the information from an external party, for example, a market research agency
- Detect discrepancies between your data and the external data at a business-relevant grouping level
