Two-Way Partial AUC and Its Properties

TL;DR

This paper introduces the two-way partial AUC, a new metric for evaluating diagnostic tests that considers restrictions on both TPR and FPR, with theoretical properties and practical applications.

Contribution

It proposes a nonparametric estimator for two-way pAUC, analyzes its asymptotic behavior, and develops a regression framework to assess covariate effects.

Findings

Estimator is asymptotically normal.

Bootstrap method effectively compares measures.

Regression framework captures covariate influences.

Abstract

When people evaluate the performance of a diagnostic test, it is important to control both True Positive Rate (TPR) and False Positive Rate (FPR). In the literature, most researchers propose the partial area under the ROC curve (pAUC) with restrictions on FPR to assess a binary classification system, which is named as FPR pAUC. It could be artificially designed to measure the area controlled by TPR and FPR, but is often misleading conceptually and practically. A new and intuitive method, named two-way pAUC, is provided in this paper, which focuses directly on the partial area under the ROC curve with both horizontal and vertical restrictions. We propose a nonparametric estimator of two-way pAUC, obtain its asymptotic normality properties and conduct the measure comparison by bootstrap method. Further, in order to evaluate possible covariate effects on two-way pAUC, regression analysis framework is constructed and corresponding theoretical properties are established. Simulation and real application are conducted to support our methods.