Building a stable classifier with the inflated argmax

TL;DR

This paper introduces a new stable classification framework that combines bagging with a novel 'inflated argmax' relaxation, ensuring robustness against data perturbations without sacrificing accuracy.

Contribution

The authors propose the 'inflated argmax' method and a stability guarantee that is distribution-free, class-independent, and applicable to any base classifier.

Findings

The inflated argmax improves classifier stability against data perturbations.

The method maintains accuracy comparable to traditional classifiers.

Stability guarantees hold without distributional assumptions.

Abstract

We propose a new framework for algorithmic stability in the context of multiclass classification. In practice, classification algorithms often operate by first assigning a continuous score (for instance, an estimated probability) to each possible label, then taking the maximizer -- i.e., selecting the class that has the highest score. A drawback of this type of approach is that it is inherently unstable, meaning that it is very sensitive to slight perturbations of the training data, since taking the maximizer is discontinuous. Motivated by this challenge, we propose a pipeline for constructing stable classifiers from data, using bagging (i.e., resampling and averaging) to produce stable continuous scores, and then using a stable relaxation of argmax, which we call the "inflated argmax," to convert these scores to a set of candidate labels. The resulting stability guarantee places no distributional assumptions on the data, does not depend on the number of classes or dimensionality of the covariates, and holds for any base classifier. Using a common benchmark data set, we demonstrate that the inflated argmax provides necessary protection against unstable classifiers, without loss of accuracy.