Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classification

TL;DR

This paper introduces an exact reformulation framework for directly optimizing precision, recall, and Fβ-score in imbalanced binary classification, outperforming existing approximation-based methods.

Contribution

It presents the first exact constrained reformulations for direct metric optimization in imbalanced classification, enabling more effective and precise model tuning.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Effective solution via exact penalty methods.

Applicable to various DMO problems beyond the studied metrics.

Abstract

For classification with imbalanced class frequencies, i.e., imbalanced classification (IC), standard accuracy is known to be misleading as a performance measure. While most existing methods for IC resort to optimizing balanced accuracy (i.e., the average of class-wise recalls), they fall short in scenarios where the significance of classes varies or certain metrics should reach prescribed levels. In this paper, we study two key classification metrics, precision and recall, under three practical binary IC settings: fix precision optimize recall (FPOR), fix recall optimize precision (FROP), and optimize $F_\beta$-score (OFBS). Unlike existing methods that rely on smooth approximations to deal with the indicator function involved, \textit{we introduce, for the first time, exact constrained reformulations for these direct metric optimization (DMO) problems}, which can be effectively solved by exact penalty methods. Experiment results on multiple benchmark datasets demonstrate the practical superiority of our approach over the state-of-the-art methods for the three DMO problems. We also expect our exact reformulation and optimization (ERO) framework to be applicable to a wide range of DMO problems for binary IC and beyond. Our code is available at https://github.com/sun-umn/DMO.