Partial AUC Maximization via Nonlinear Scoring Functions

TL;DR

This paper introduces nonlinear scoring functions, including generative models and deep neural networks, to directly maximize the partial AUC in binary classification, demonstrating improved performance over traditional linear methods in astronomical data analysis.

Contribution

It proposes novel nonlinear scoring functions for pAUC maximization, extending beyond conventional linear approaches, with applications in astronomy.

Findings

Nonlinear scoring functions outperform linear methods in pAUC maximization.

Deep neural network-based scoring functions show significant improvement.

Application to astronomical data validates the effectiveness of the proposed methods.

Abstract

We propose a method for maximizing a partial area under a receiver operating characteristic (ROC) curve (pAUC) for binary classification tasks. In binary classification tasks, accuracy is the most commonly used as a measure of classifier performance. In some applications such as anomaly detection and diagnostic testing, accuracy is not an appropriate measure since prior probabilties are often greatly biased. Although in such cases the pAUC has been utilized as a performance measure, few methods have been proposed for directly maximizing the pAUC. This optimization is achieved by using a scoring function. The conventional approach utilizes a linear function as the scoring function. In contrast we newly introduce nonlinear scoring functions for this purpose. Specifically, we present two types of nonlinear scoring functions based on generative models and deep neural networks. We show experimentally that nonlinear scoring fucntions improve the conventional methods through the application of a binary classification of real and bogus objects obtained with the Hyper Suprime-Cam on the Subaru telescope.