Revisiting sample size planning for receiver operating characteristic studies: a confidence interval approach with precision and assurance

TL;DR

This paper develops and evaluates simple sample size formulas for estimating AUCs in diagnostic studies, focusing on achieving desired precision and assurance levels, with validation through simulations and real-world examples.

Contribution

It introduces new sample size formulas explicitly incorporating precision and assurance, accounting for unequal variances, and provides an online calculator for practical use.

Findings

Formulas achieve empirical assurance close to target levels

Coverage probabilities are close to the nominal 95%

Simulation confirms good performance of the formulas

Abstract

Objectives: Estimation of areas under receiver operating characteristic curves (AUCs) and their differences is a key task in diagnostic studies. We aimed to derive, evaluate, and implement simple sample size formulas for such studies with a focus on estimation rather than hypothesis testing. Materials and Methods: Sample size formulas were developed by explicitly incorporating pre-specified precision and assurance, with precision denoted by the lower limit of confidence interval and assurance denoted by the probability of achieving that lower limit. A new variance function was proposed for valid estimation allowing for unequal variances of observations in the disease and non-disease groups. Performance of the proposed formulas was evaluated through simulation. Results: Closed-form sample size formulas were obtained. Simulation results demonstrated that the proposed formulas produced empirical assurance probability close to the pre-specified assurance probability and empirical coverage probability close to the nominal 95%. Real-world worked examples were presented for illustration. Conclusions: Sample size formulas based on estimation of AUCs and their differences were developed. Simulation results suggested good performance in terms of achieving pre-specified precision and assurance probability. An online calculator for implementing the proposed formulas is openly available at https://dishu.page/calculator/.