Country-specific estimates of misclassification rates of computer-coded verbal autopsy algorithms

TL;DR

This paper evaluates how well computer algorithms classify causes of death in different countries and proposes a method to improve their accuracy using country-specific data.

Contribution

The study introduces country-specific misclassification rate estimates for CCVA algorithms, improving CSMF accuracy without requiring CHAMPS data.

Findings

Country-specific models reduce misclassification loss by 34%–38% for neonates and 13%–24% for children compared to homogeneous models.

CCVA algorithms consistently overestimate or underestimate certain causes of death.

Calibrating data increases neonatal CSMF for infections and decreases it for intrapartum events and prematurity.

Abstract

Computer-coded verbal autopsy (CCVA) algorithms are routinely used to determine individual cause of death (COD) and derive population-level estimates of cause-specific mortality fractions (CSMFs). But frequent COD misclassification leads to biased CSMF estimates. The VA-calibration framework reduces the bias by estimating misclassification rates; but it overlooks systematic patterns and cross-country variation, reducing the accuracy of CSMF estimates.

Using CHAMPS (Child Health and Mortality Prevention Surveillance) data and the framework in Pramanik et al (2025), we estimate misclassification rates of three widely used CCVA algorithms (Expert Algorithm VA, InSilicoVA and InterVA), two age groups (neonates aged 0–27 days and children aged 1–59 months), and eight countries (Bangladesh, Ethiopia, Kenya, Mali, Mozambique, Sierra Leone, South Africa and ‘other’). We then demonstrate their utility and use the Mozambique-specific rates to calibrate VA-only data from the Countrywide Mortality Surveillance for Action (COMSA) project in Mozambique.

We report three key findings. First, the country-specific model better fits CHAMPS misclassification rates than the homogeneous model, reducing average absolute loss by 34%–38% for neonates and 13%–24% for children. Second, CCVA algorithms show consistent misclassification patterns, systematically overestimating or underestimating certain causes. Third, calibrating COMSA data increases neonatal CSMF for sepsis/meningitis/infection and decreases it for intrapartum-related events and prematurity; among children, CSMF increases for malaria and decreases for pneumonia.

We present an inventory of VA misclassification rate estimates across two age groups, three CCVA algorithms and eight countries. These publicly available estimates enable the calibration of VA-only data from any country without needing access to CHAMPS data. More generally, these analyses reveal systematic algorithmic biases and highlight opportunities to refine future CCVA algorithms. As reliance on computer-coded and AI-driven approaches to COD determination grows, our integrated VA-calibration workflow, grounded in robust statistical frameworks and open-source software (misclassification matrix modeling, VA-calibration R package on GitHub and CRAN), offers a critical step towards improving the accuracy of mortality surveillance.

Figures