Bayesian Receiver Operating Characteristic Metric for Linear Classifiers

TL;DR

This paper introduces a Bayesian AUC metric for linear classifiers that estimates accuracy directly from training data, eliminating the need for cross-validation and providing faster, more accurate assessments.

Contribution

It presents a closed-form Bayesian AUC estimator for linear classifiers under Gaussian assumptions, improving speed and accuracy over traditional methods.

Findings

The closed-form CBAUC is faster than conventional estimators.

CBAUC provides more accurate accuracy estimates.

Experiments confirm effectiveness on simulated and real data.

Abstract

We propose a novel classifier accuracy metric: the Bayesian Area Under the Receiver Operating Characteristic Curve (CBAUC). The method estimates the area under the ROC curve and is related to the recently proposed Bayesian Error Estimator. The metric can assess the quality of a classifier using only the training dataset without the need for computationally expensive cross-validation. We derive a closed-form solution of the proposed accuracy metric for any linear binary classifier under the Gaussianity assumption, and study the accuracy of the proposed estimator using simulated and real-world data. These experiments confirm that the closed-form CBAUC is both faster and more accurate than conventional AUC estimators.