Measuring Diagnostic Test Performance Using Imperfect Reference Tests: A Partial Identification Approach

TL;DR

This paper develops a method to estimate the true performance of diagnostic tests when the reference standard is imperfect, providing bounds on sensitivity, specificity, and related parameters using partial identification techniques.

Contribution

It introduces a novel partial identification approach to bound true diagnostic test performance metrics without assuming a perfect reference test.

Findings

Bounds on sensitivity and specificity can be very informative.

Estimated false negative rates are significantly higher than apparent rates.

Method applied to COVID-19 antigen test data demonstrates practical utility.

Abstract

Diagnostic tests are almost never perfect. Studies quantifying their performance use knowledge of the true health status, measured with a reference diagnostic test. Researchers commonly assume that the reference test is perfect, which is often not the case in practice. When the assumption fails, conventional studies identify "apparent" performance or performance with respect to the reference, but not true performance. This paper provides the smallest possible bounds on the measures of true performance - sensitivity (true positive rate) and specificity (true negative rate), or equivalently false positive and negative rates, in standard settings. Implied bounds on policy-relevant parameters are derived: 1) Prevalence in screened populations; 2) Predictive values. Methods for inference based on moment inequalities are used to construct uniformly consistent confidence sets in level over a relevant family of data distributions. Emergency Use Authorization (EUA) and independent study data for the BinaxNOW COVID-19 antigen test demonstrate that the bounds can be very informative. Analysis reveals that the estimated false negative rates for symptomatic and asymptomatic patients are up to 3.17 and 4.59 times higher than the frequently cited "apparent" false negative rate. Further applicability of the results in the context of imperfect proxies such as survey responses and imputed protected classes is indicated.