Supervised classification via minimax probabilistic transformations

TL;DR

This paper introduces linear probabilistic classifiers (LPCs) based on robust risk minimization, which optimize 0-1 loss directly, handle unconstrained rules, and provide performance guarantees, showing competitive results on benchmark datasets.

Contribution

The paper proposes LPCs that optimize 0-1 loss directly using RRM, allowing unconstrained classification rules and efficient learning with theoretical performance bounds.

Findings

LPCs achieve competitive accuracy on benchmark datasets.

LPCs provide finite-sample generalization bounds.

LPCs avoid overfitting through RRM over polyhedral uncertainty sets.

Abstract

Conventional techniques for supervised classification constrain the classification rules considered and use surrogate losses for classification 0-1 loss. Favored families of classification rules are those that enjoy parametric representations suitable for surrogate loss minimization, and low complexity properties suitable for overfitting control. This paper presents classification techniques based on robust risk minimization (RRM) that we call linear probabilistic classifiers (LPCs). The proposed techniques consider unconstrained classification rules, optimize the classification 0-1 loss, and provide performance bounds during learning. LPCs enable efficient learning by using linear optimization, and avoid overffiting by using RRM over polyhedral uncertainty sets of distributions. We also provide finite-sample generalization bounds for LPCs and show their competitive performance with state-of-the-art techniques using benchmark datasets.