All resources
Preventing an error in data collection is easier than dealing with its consequences. The sagacity of your business decisions depends on the quality of your data. In this article, we tell you about data quality checks at all collection stages, from the statement of work to completed reports.
It’s crucial to assess data quality early on and ensure data quality monitoring and data quality check by using effective data quality testing tools and techniques that examine data against established quality dimensions and uncover inconsistencies.
Data engineering plays a critical role in ensuring data quality throughout the data pipeline. Data engineers are essential in managing data quality issues and implementing essential data quality tests.
A comprehensive data quality assessment is essential in the early stages of data collection, employing various techniques such as: data quality testing, profiling, validation, and cleansing to establish specific quality rules and thresholds.
Want to be sure about the data quality status of your data? Leave it to OWOX BI. We’ll help you develop data quality metrics for the data team and customize your analytics processes to ensure high data quality throughout the data collection and data quality monitoring process.
With OWOX BI, tracking data quality metrics for data quality becomes a primary key component of our service, ensuring the accuracy of your data. You don’t need to look for connectors and clean up and process data. You’ll get ready data sets in an understandable and easy-to-use structure.
Note: This post was originally published in January 2020 and was completely updated in March 2025 for accuracy and comprehensiveness.
Data quality refers to data conditions in terms of accuracy, completeness, reliability, and relevance. It is essential for making informed decisions, driving business efficiency, and ensuring compliance with regulations. Poor data quality can lead to incorrect insights, operational inefficiencies, and financial losses.
By understanding and maintaining data quality, organizations can ensure that their data is suitable for analysis and decision-making, forming the backbone of successful data-driven strategies.
Data quality check is crucial for organizations to ensure data accuracy, consistency, and reliability. Key benefits to improve data quality can include:
The data quality testing process is essential for ensuring precise and reliable data management, leveraging automation and advanced technologies, and aligning data quality with organizational needs through collaboration with business stakeholders.
To effectively monitor and improve these aspects, organizations rely on data quality metrics, quantitative indicators that track and report changes in data quality dimensions and issues over time.
It’s crucial to track data quality metrics for effective data quality check, as they serve as quantitative indicators to determine the accuracy of data, enabling organizations to find areas of improvement and confirm the effectiveness of monitoring tools.
The data quality monitoring process is a continuous one of evaluating and ensuring the integrity, accuracy, and reliability of data throughout its lifecycle, beginning with a foundational step of data quality assessment.
This assessment involves context awareness and the application of various techniques such as data profiling, data validation, and data cleansing, along with the establishment of specific data quality rules and thresholds. It sets the stage for effective monitoring by identifying the key metrics to track.
The process of data quality monitoring involves setting benchmarks for data quality attributes such as completeness, consistency, and timeliness and using tools and methodologies, including advanced data quality monitoring techniques, to track data quality metrics against these standards.
By identifying anomalies and errors in real-time, data quality monitoring enables organizations to take immediate corrective actions, thereby preventing the negative impacts of low-quality data on business operations and decision-making.
The importance of choosing a data quality monitoring solution that enables quick identification and resolution of data quality issues cannot be overstated, as they are vital for preserving the overall health of the data ecosystem.
Monitoring the quality of your data is crucial for ensuring its reliability and usability in decision-making processes. Addressing poor data quality issues early helps prevent errors and reduces costs associated with inaccuracies, enhancing operational efficiency. An effective data quality strategy is essential for improving and maintaining high data quality standards within organizations.
Regular data quality monitoring safeguards against the potential risks of regulatory non-compliance and protects the organization’s reputation from the negative impacts of data quality issues.
Tracking data quality metrics is essential in this process, as it provides quantitative indicators that help determine the accuracy of data, thereby enhancing the decision-making process by providing accurate data insights.
Through upholding rigorous data standards and consistently monitoring and rectifying data quality concerns, companies can gain deeper insights into their performance, enhance customer interactions, and secure a competitive advantage in the market, resulting in enhanced business results.
Data quality challenges are common issues that organizations face when managing their data assets. These challenges can significantly impact the reliability and usability of data, leading to incorrect decisions and operational inefficiencies. Some common data quality challenges include:
Addressing these data quality issues requires a proactive approach to data quality management, including regular data quality checks, validation, and cleansing processes.
💡 Learn how to maintain data quality with our detailed guide, Common Data Quality Issues and How to Overcome Them. Explore practical solutions to address common challenges and ensure accurate, reliable insights for better decision-making.
Unfortunately, many companies that spend substantial resources storing and processing data still make important decisions based on intuition, human error, and their own expectations instead of data. Utilizing data performance testing tools is essential for simulating various data processing scenarios to ensure the reliability of web analytics. Additionally, data quality tests play a crucial role in evaluating and ensuring the reliability of data across various processes and systems.
Why does that happen? Distrust of data is exacerbated by situations where data provides an answer that’s at odds with the expectations of the decision-maker. In addition, if someone has encountered errors in data or reports in the past, they’re inclined to favor intuition. This is understandable, as a decision made on the basis of incorrect or incomplete data may throw you back rather than move you forward.
Imagine you have a multi-currency project. Your analyst has set up Google Analytics in one currency, and the marketer in charge of contextual advertising has set up cost importing into Google Analytics 4 in another currency. As a result, you have an unrealistic return on ad spend (ROAS) in your advertising campaign reports. If you don’t notice this error in time, you may either disable profitable campaigns or increase the budget on loss-making ones.
In addition, developers are usually very busy, and implementing web analytics is a secondary task for them. While implementing new functionality — for example, a new design for a unit with accessories — developers may forget to check that data is being collected in Google Analytics 4. As a result, when the time comes to evaluate the effectiveness of the new design, it turns out that the data collection was broken two weeks ago. Surprise.
We recommend testing web analytics data as early and as often as possible to minimize the cost of correcting an error.
Imagine you’ve made an error during the specification phase. If you find it and correct it immediately, the fix will be relatively cheap. If the error is revealed after implementation, when building reports, or even when making decisions, the cost of fixing it will be very high.
Data quality testing plays a crucial role in preventing such costly errors and ensuring data accuracy.
Data collection typically consists of five primary key steps:
At almost all of these stages, it’s very important to check your data. It’s necessary to test technical documentation, Google Analytics 4 and Google Tag Manager settings, and, of course, the quality of data collected on your site or in your mobile application. Monitoring and testing at various stages of the data pipeline are crucial to ensure data quality and reliability.
Before you go to each step, let's take a look at some requirements for data testing:
As we've mentioned, it's much easier to correct an error if you catch it in the specifications. Therefore, checking documentation starts long before collecting data. Let's figure out why we need to check your documentation.
Data validation is crucial in preventing these errors by ensuring the data meets established quality criteria before it's processed.
The next step after you check your technical documentation is to check your Google Analytics 4 and Google Tag Manager settings.
Why test Google Analytics 4 and Google Tag Manager settings?
Most common errors in Google Analytics:
Most common errors in Google Tag Manager:
The last stage of testing is testing directly on the site. This stage requires more technical knowledge because you must watch the code, check how the container is installed, and read the logs. So, you need to be savvy and use the right tools.
Why test embedded metrics?
The most common mistakes:
Common data quality issues are problems that organizations face when managing their data assets. These issues can compromise the integrity and reliability of data, leading to incorrect analysis and decision-making. Some common data quality issues include:
By addressing these common data quality issues, organizations can ensure their data is reliable, accurate, and fit for purpose.
Maintaining high-quality data is essential for accurate analysis and decision-making. Implementing best practices helps organizations ensure data consistency, reliability, and accuracy, enabling seamless operations and better outcomes across all business functions.
Validating your findings ensures data accuracy by comparing results to expected outcomes. This process helps identify errors, inconsistencies, or anomalies during data profiling. Using validation rules and automated checks strengthens data reliability and supports accurate decision-making.
Educating users about data quality standards and practices ensures better adherence to governance policies. Training sessions and awareness programs empower teams to identify issues, follow best practices, and maintain data consistency, enhancing data reliability and accuracy.
Ensuring data quality starts with verifying the reliability of data sources. Regularly assess source accuracy and trustworthiness to prevent errors from propagating through systems, ensuring consistent, dependable data for analysis and decision-making processes.
Standardizing data formats and structures ensures consistency across datasets, enabling smoother integration and analysis. By applying uniform standards, organizations reduce errors, simplify data processing, make data transformations, and enhance collaboration, ensuring data quality for reliable decision-making and efficient operations.
Regular data audits help identify errors, inconsistencies, and outdated records, ensuring data accuracy and reliability. These audits enable organizations to maintain high-quality data, streamline processes, and support effective decision-making across all business functions.
Data quality Tools we use to test data:
Let's take a closer look at these tools. Data quality tools are essential for generating data quality metrics and applying data quality rules to ensure data accuracy, consistency, and reliability.
To get started, you need to install this extension in your browser and enable it. Then open the page ID and go to the Console tab. The extension provides the information you see.
This screen shows the parameters that are transmitted with hits and the values that are transmitted for those parameters:
There's also an extended e-commerce block. You can find it in the console as ec:
In addition, error messages are displayed here, such as for exceeding the hit size limit.
If you need to check the composition of the dataLayer, the easiest way to do this is to type the dataLayer command in the console:
Here are all the parameters that are transmitted. You can study them in detail and verify them. Each action on the site is reflected in the dataLayer. Let's say you have seven objects. If you click on an empty field and call the dataLayer command again, an eighth object should appear in the console.
To access Google Tag Manager Debugger, open your Google Tag Manager account and click the Preview button:
Then, open your site and refresh the page. In the lower pane, a panel should appear that shows all the tags running on that page.
Events that are added to the dataLayer are displayed on the left. By clicking on them, you can check the real-time composition of the dataLayer.
Features of mobile browser testing:
Features of mobile application testing:
This step is the fastest and easiest. At the same time, it makes sure the data collected in Google Analytics 4 makes sense. In your reports, you can check hundreds of different scenarios and look at indicators depending on the device, browser, etc. If you find any anomalies in the data, you can play the script on a specific device and in a specific browser.
You can also use Google Analytics 4 reports to check the completeness of data transferred to the data layer. That is, depending on each of the scenarios, the variable is filled, whether there are all parameters in it, whether the parameters take the correct values, etc.
We want to share the most useful reports (in our opinion). You can use them as a data collection checklist:
Let's see what these reports look like in the interface and which of these reports you need to pay attention to first.
In GA4, the "E-commerce purchases" report not only tracks user progression through different stages of the shopping journey but also helps in assessing the completeness of data collection at each stage. This is crucial for identifying any gaps where data might not be accurately captured.
For instance, if a significant drop-off is observed between the "add to cart" and "purchase" stages, it could indicate issues with the checkout process or with how events are tracked in that segment.
GA4 uses event-based tracking, which offers flexibility in monitoring specific interactions across the site. Designated events track each stage of the enhanced e-commerce process – viewing products, adding items to carts, initiating checkout, and completing a purchase.
Analyzing these events can reveal discrepancies or inefficiencies in data collection, enabling marketers to adjust tracking setups or site design to ensure comprehensive data collection and a smoother user experience.
What should we pay attention to here? First, it's very strange if you have zero values in any of the columns. Second, if you have more values at some stage than the previous stage, you'll likely need help collecting data.
That's weird and worth paying attention to. You can also switch between other parameters in this report, which should also be sent to Enhanced E-commerce.
First of all, it's necessary to walk through all parameters that are transmitted to Google Analytics and see what values each parameter takes. Usually, it's immediately clear whether everything is okay. More detailed analyses of each of the events can be carried out in custom reports.
Cost Analysis is another report that can be useful for checking expense data importing into Google Analytics.
We often see reports with expenses for some source or advertising campaign but no sessions. Problems or errors in UTM tags can cause this. Alternatively, filters in Google Analytics 4 may exclude sessions from a particular source. These reports need to be checked from time to time.
We would like to highlight the custom report that allows you to track duplicate transactions. It's very easy to set up: the parameter must be a transaction ID, and the key dimension must be transactions.
Note that when there's more than one transaction in the report, information about the same order is sent more than once.
If you find a similar problem, read these detailed instructions on how to fix it.
Google Analytics has a very good Custom Alerts tool that allows you to track important changes without viewing reports. For example, if you stop collecting information about Google Analytics sessions, you can receive an email notification.
We recommend that you set up notifications for at least these four metrics:
In our experience, this is the most difficult and time-consuming task — the narrow line where mistakes are the most common.
To avoid problems with dataLayer implementation, checks must be done at least once a week. In general, the frequency should depend on how often you implement changes on the site. Ideally, you need to test the dataLayer after each significant change. Doing this manually is time-consuming, so we decided to automate the process.
To automate testing, we've built a cloud-based solution that enables us to:
Advantages of test automation:
A simplified scheme of the algorithm we use:
When you sign in to our app, you need to specify the pages you want to verify. You can do this by uploading a CSV file, specifying a link to the sitemap, or simply specifying a site URL, in which case the application will find the sitemap itself.
Then, it's important to specify the dataLayer scheme for each scenario to be tested: pages, events, scripts (a sequence of actions, such as for checkout). Then, you can use regular expressions to specify that the page types match the URL.
After receiving all this information, our application runs through all pages and events as scheduled, checks each script, and uploads test results to Google BigQuery. Based on this data, we set up email and Slack notifications.
