monival.org

04 · Data quality, sampling and collection

Data quality assessment (DQA)

A data quality assessment (DQA) is a structured exercise that checks whether reported results can be trusted: it verifies reported numbers against source records and appraises the data-management system that produced them. The standard toolkits assess a common core of dimensions — completeness, timeliness, internal and external consistency, and accuracy — and end in a costed action plan, not just a score.

Last updated · Reviewed against 4 cited sources

Why assess data quality at all

Every indicator value in a results report is the end of a chain: someone recorded an event in a source document, someone tallied it into a register, someone aggregated it into a monthly report, and someone entered that into an information system. Errors enter at every link — omission, double counting, transcription, late reporting, definition drift — and aggregation hides them. A data quality assessment is the discipline of walking that chain deliberately, measuring how much the reported numbers can be trusted, and fixing the system rather than the symptom [1].

A typical reporting chain, from source documents to national figuresA stack of five layers. From bottom to top: source documents such as registers and forms; facility or site monthly summaries; district aggregation; programme or national information system; published indicator values. Errors introduced low in the stack propagate upward and are hidden by aggregation.Published indicator valueNational / programme systemelectronic aggregation, dashboardsDistrict aggregationcollation, re-entryFacility / site reportmonthly tally and summarySource documentsregisters, forms, case files — where truth lives
Figure 1. Verification works down this stack; each layer inherits — and conceals — the errors of the layers beneath it.

What a DQA measures

The WHO Data Quality Review framework and the MEASURE Evaluation tool family converge on a common core of dimensions [1][2]:

  • Completeness — are all expected reports present, and are the data elements within them filled in?
  • Timeliness — did reports arrive by the deadline that makes them usable?
  • Internal consistency — do the data agree with themselves: no impossible outliers, plausible trends, agreement between related indicators, and agreement between reported and recounted values?
  • External consistency (accuracy) — do the data agree with independent sources such as surveys or a parallel system?

The last check under internal consistency — reported versus recounted — is data verification, and it is the heart of any DQA:

The data verification procedureFive steps in sequence: select indicators and sites; trace reported figures to source documents; recount results from the sources; compare the recount to the reported figure to compute a verification factor; then explain discrepancies and agree corrective actions.Selectindicators & sitesTracereport → sourcesRecountfrom source documentsCompareverification factorActexplain & correct
Figure 2. The verification factor (recounted ÷ reported) quantifies over-reporting (below 1) or under-reporting (above 1) at each level of the chain.

Alongside verification, the systems assessment appraises whether the data-management machinery — roles, definitions, forms, training, supervision, storage, transmission — is capable of producing good data routinely. A perfect recount from a system held together by one heroic records officer is not quality; it is luck with a shelf life [3].

Audit or routine? Choose deliberately

The tool family splits along two axes: who assesses, and how often.

Data quality assessment modalities by assessor and cadenceA two-by-two matrix. The horizontal axis runs from one-off to routine cadence; the vertical axis from internal to external assessor. External plus one-off is the data quality audit; internal plus routine is the RDQA self-assessment; external plus routine is ongoing supervision with spot checks; internal plus one-off is a pre-audit readiness self-check.Data Quality Audit (DQA)external team certifiesreported results;credibility for fundersSupervision & spot checksexternal programmeofficers verify duringroutine visitsReadiness self-checkinternal dry run before anexternal auditRoutine DQA (RDQA)self-assessment thatbuilds capacity andcatches drift earlyCadenceone-off →→ routineAssessorinternal →→ external
Figure 3. Audit and self-assessment are complements: the audit certifies, the routine cycle improves.Distinction follows the MEASURE Evaluation DQA and RDQA guidelines.

The external audit variant exists to give funders and governments confidence in reported results [4]; the routine variant (RDQA) exists to build the habits that make the audit boring [3]. Programmes that only ever audit stay bad at data; programmes that only self-assess struggle to be believed.

The RDQA as a cycle, not an event

Run routinely, a DQA is a management loop:

The routine data quality assessment cycleSix steps arranged in a loop: plan and select indicators; desk review of completeness, timeliness and consistency; site verification of reported against recounted results; systems assessment; action plan with owners and dates; follow up and repeat the cycle.01Plan & selectindicators02Desk review03Siteverification04Systemsassessment05Action plan06Follow up &repeatquality as aroutine, not arescue
Figure 4. The action plan — owners, deadlines, follow-up — is the step that separates assessment from theatre.

Two practical rules keep the cycle honest. First, publish the verification factors, including the bad ones; a DQA whose results are quietly shelved teaches every site that accuracy is optional. Second, track action-plan completion as an indicator in its own right — the percentage of last cycle’s corrective actions closed is the single best predictor of whether data quality is actually improving.

Checklist before you run one

  • Indicators selected are the ones decisions actually ride on, not the easiest to count.
  • The full reporting chain for each indicator is mapped before fieldwork.
  • Source documents are defined precisely enough that two assessors would recount the same number.
  • The systems-assessment checklist is scored on evidence, not assertion.
  • Every finding lands in an action plan with an owner and a date — and the next cycle opens by reviewing it.

Sources

  1. Data Quality Assurance. Module 1: Framework and metrics — World Health Organization, 2017.The DQR framework: dimensions, standard metrics, and the desk-review / verification split.
  2. Data Quality Tools (DQA and RDQA tool family) — MEASURE Evaluation (USAID), 2017.Index of the audit (DQA) and routine (RDQA) instruments, guidelines and workbooks.
  3. Routine Data Quality Assessment Tool: User Manual — MEASURE Evaluation (USAID), 2017.Operational manual for the RDQA: verification steps, system checklist, dashboards and action planning.
  4. Data Quality Audit Tool: Guidelines for Implementation — MEASURE Evaluation (USAID), 2008.The external-audit variant: protocols for audit teams verifying reported programme results.