Optimal ROC Curves from Score Variable Threshold Tests

TL;DR

This paper proves that concavity of ROC curves generated by score variable threshold tests (SVT's) is both necessary and sufficient for Neyman-Pearson optimality, and provides a method to transform non-concave SVT ROC into likelihood ratio test (LRT) ROC.

Contribution

It establishes the sufficiency of concavity for SVT-generated ROC optimality and offers a constructive method to derive the LRT ROC from non-concave SVT ROC without needing explicit PDFs.

Findings

Concavity of ROC from SVT's is necessary and sufficient for Neyman-Pearson optimality.

A procedure to generate the LRT ROC from any non-concave SVT ROC.

Method to redesign tests to match LRT performance if PDFs are known.

Abstract

The Receiver Operating Characteristic (ROC) is a well-established representation of the tradeoff between detection and false alarm probabilities in binary hypothesis testing. In many practical contexts ROC's are generated by thresholding a measured score variable -- applying score variable threshold tests (SVT's). In many cases the resulting curve is different from the likelihood ratio test (LRT) ROC and is therefore not Neyman-Pearson optimal. While it is well-understood that concavity is a necessary condition for an ROC to be Neyman-Pearson optimal, this paper establishes that it is also a sufficient condition in the case where the ROC was generated using SVT's. It further defines a constructive procedure by which the LRT ROC can be generated from a non-concave SVT ROC, without requiring explicit knowledge of the conditional PDF's of the score variable. If the conditional PDF's are known, the procedure implicitly provides a way of redesigning the test so that it is equivalent to an LRT.