Uncertainties in ROC (Receiver Operating Characteristic) Curves Derived from Counting Data

TL;DR

This paper examines how to properly assign uncertainties to ROC curves derived from counting data, addressing a longstanding unresolved issue in the evaluation of decision-making systems.

Contribution

It provides a practical operational approach to quantifying uncertainties in ROC curves specifically for counting experiment data.

Findings

Clarifies the concept of uncertainty in ROC curves from counting data

Proposes a practical method for uncertainty estimation in ROC analysis

Addresses a historical gap in ROC curve interpretation

Abstract

The ROC (receiver operating characteristic) curve is a widely used device for assessing decision-making systems. It seems surprising, in view of its history dating back to World War Two, that the assignment of uncertainties to a ROC curve is apparently not settled. This note returns to the question, focusing on the application of ROC curves to the analysis of data from counting experiments and taking a practical operational approach to the concept of uncertainty.