The cost of not having a perfect reference in diagnostic accuracy studies: theoretical results and a web visualisation tool

TL;DR

This paper analyzes the impact of imperfect reference standards on diagnostic accuracy studies, providing theoretical insights and a web visualization tool to compare methods like imperfect gold standards, discrepant analysis, and latent class models.

Contribution

It offers a theoretical comparison of methods for evaluating diagnostic tests without perfect references, including an interactive visualization tool.

Findings

Deviations depend on test sensitivities, specificities, prevalence, and dependence.

Theoretical expressions quantify biases introduced by different methods.

The visualization tool aids in understanding method impacts under various conditions.

Abstract

Dichotomous diagnostic tests are widely used to detect the presence or absence of a biomedical condition of interest. A rigorous evaluation of the accuracy of a diagnostic test is critical to determine its practical value. Performance measures, such as the sensitivity and specificity of the test, should be estimated by comparison with a gold standard. Since an error-free reference test is frequently missing, approaches based on available imperfect diagnostic tests are used, namely: comparisons with an imperfect gold standard or with a composite reference standard, discrepant analysis, and latent class models. In this work, we compare these methods using a theoretical approach based on analytical expressions for the deviations between the sensitivity and specificity according to each method, and the corresponding true values. We explore the impact on the deviations of varying conditions: tests sensitivities and specificities, prevalence of the condition and local dependence between the tests. An R interactive graphical application is made available for the visualisation of the outcomes. Based on our findings, we discuss the methods validity and potential usefulness.