Reducing decision errors in the paired comparison of the diagnostic accuracy of screening tests with Gaussian outcomes

TL;DR

This paper introduces a weighted maximum likelihood bias correction method to improve decision accuracy in paired comparisons of diagnostic tests with Gaussian outcomes, especially when bias affects the area under the ROC curve.

Contribution

The paper presents a novel bias correction method for paired ROC analysis, reducing decision errors in diagnostic accuracy studies with Gaussian outcomes.

Findings

Bias correction reduces Type I error rate in simulations.

Method improves power for correct decision-making.

Performance depends on test characteristics and gold standard follow-up percentage.

Abstract

Scientists often use a paired comparison of the areas under the receiver operating characteristic curves to decide which continuous cancer screening test has the best diagnostic accuracy. In the paired design, all participants are screened with both tests. Participants with unremarkable screening results enter a follow-up period. Participants with suspicious screening results and those who show evidence of disease during follow-up receive the gold standard test. The remaining participants are classified as non-cases, even though some may have occult disease. The standard analysis includes all study participants in the analysis, which can create bias in the estimates of diagnostic accuracy. If the bias affects the area under the curve for one screening test more than the other screening test, scientists may make the wrong decision as to which screening test has better diagnostic accuracy. We describe a weighted maximum likelihood bias correction method to reduce decision errors. We assessed the ability of the bias correction method to reduce decision errors via simulation studies. The simulations compared the Type I error rate and power of the standard analysis with that of the bias-corrected analysis. The performance of the bias correction method depends on characteristics of the screening tests and the disease, and on the percentage of study participants who receive the gold standard test. In studies with a large amount of bias in the difference in the full area under the curve, the bias correction method reduces the Type I error rate and improves power for the correct decision. In order to determine if bias correction is needed for a specific screening trial, we recommend the investigator conduct a simulation study using our free software.