Classification Under Uncertainty: Data Analysis for Diagnostic Antibody Testing

TL;DR

This paper introduces a novel decision-theoretic approach for classification in diagnostic antibody testing that explicitly accounts for disease prevalence and measurement uncertainty, significantly reducing errors compared to traditional methods.

Contribution

It develops an optimal classification framework that incorporates prevalence and uncertainty, with adaptive and hold-out strategies for unknown prevalence, and extends ROC analysis through optimization.

Findings

Reduces classification errors by up to a decade.

Provides a theoretical foundation linking ROC to optimization.

Demonstrates effectiveness on SARS-CoV-2 serology data.

Abstract

Formulating accurate and robust classification strategies is a key challenge of developing diagnostic and antibody tests. Methods that do not explicitly account for disease prevalence and uncertainty therein can lead to significant classification errors. We present a novel method that leverages optimal decision theory to address this problem. As a preliminary step, we develop an analysis that uses an assumed prevalence and conditional probability models of diagnostic measurement outcomes to define optimal (in the sense of minimizing rates of false positives and false negatives) classification domains. Critically, we demonstrate how this strategy can be generalized to a setting in which the prevalence is unknown by either: (i) defining a third class of hold-out samples that require further testing; or (ii) using an adaptive algorithm to estimate prevalence prior to defining classification domains. We also provide examples for a recently published SARS-CoV-2 serology test and discuss how measurement uncertainty (e.g. associated with instrumentation) can be incorporated into the analysis. We find that our new strategy decreases classification error by up to a decade relative to more traditional methods based on confidence intervals. Moreover, it establishes a theoretical foundation for generalizing techniques such as receiver operating characteristics (ROC) by connecting them to the broader field of optimization.