A Unified Inference Method for FROC-type Curves and Related Summary Indices

TL;DR

This paper introduces a unified statistical method for accurately estimating and evaluating FROC curves and related indices, enhancing the assessment of detection accuracy in medical imaging tasks.

Contribution

A novel maximum likelihood-based approach that simultaneously estimates smooth FROC curves and confidence intervals for summary metrics, applicable to medical device performance evaluation.

Findings

The method provides reliable confidence intervals for FROC indices.

Simulation studies demonstrate good finite sample performance.

Applied to SaMD for pulmonary lesion detection, showing practical utility.

Abstract

Free-response observer performance studies are of great importance for accuracy evaluation and comparison in tasks related to the detection and localization of multiple targets or signals. The free-response receiver operating characteristic (FROC) curve and many similar curves based on the free-response observer performance assessment data are important tools to display the accuracy of detection under different thresholds. The true positive rate at a fixed false positive rate and summary indices such as the area under the FROC curve are also commonly used as the figures of merit in the statistical evaluation of these studies. Motivated by a free-response observer performance assessment research of a Software as a Medical Device (SaMD), we propose a unified method based on the initial-detection-and-candidate model to simultaneously estimate a smooth curve and derive confidence intervals for summary indices and the true positive rate at a fixed false positive rate. A maximum likelihood estimator is proposed and its asymptotic normality property is derived. Confidence intervals are constructed based on the asymptotic normality of our maximum likelihood estimator. Simulation studies are conducted to evaluate the finite sample performance of the proposed method. We apply the proposed method to evaluate the diagnostic performance of the SaMD for detecting pulmonary lesions.