Efficient Large-Scale Learning of Minimax Risk Classifiers

TL;DR

This paper introduces an efficient algorithm for large-scale minimax risk classifiers that significantly speeds up training on big datasets with many classes, making such models more practical.

Contribution

It presents a novel combination of constraint and column generation techniques to enable scalable learning of minimax risk classifiers for multi-class problems.

Findings

Up to 10x faster training on large datasets.

Approximately 100x speedup with many classes.

Effective on multiple benchmark datasets.

Abstract

Supervised learning with large-scale data usually leads to complex optimization problems, especially for classification tasks with multiple classes. Stochastic subgradient methods can enable efficient learning with a large number of samples for classification techniques that minimize the average loss over the training samples. However, recent techniques, such as minimax risk classifiers (MRCs), minimize the maximum expected loss and are not amenable to stochastic subgradient methods. In this paper, we present a learning algorithm based on the combination of constraint and column generation that enables efficient learning of MRCs with large-scale data for classification tasks with multiple classes. Experiments on multiple benchmark datasets show that the proposed algorithm provides upto a 10x speedup for general large-scale data and around a 100x speedup with a sizeable number of classes.