Improved Multi-Class Cost-Sensitive Boosting via Estimation of the Minimum-Risk Class

TL;DR

This paper introduces a unified multi-class cost-sensitive boosting framework that directly estimates the minimum-risk class, optimizing weak learners for important class distinctions and achieving state-of-the-art results across diverse datasets.

Contribution

The paper proposes a novel boosting method that directly estimates the minimum-risk class, with a new loss function, theoretical proof of boostability, and an efficient decision tree training procedure.

Findings

Achieves state-of-the-art performance on multiple datasets

Effectively focuses on discriminating important classes

Demonstrates theoretical boostability of the algorithm

Abstract

We present a simple unified framework for multi-class cost-sensitive boosting. The minimum-risk class is estimated directly, rather than via an approximation of the posterior distribution. Our method jointly optimizes binary weak learners and their corresponding output vectors, requiring classes to share features at each iteration. By training in a cost-sensitive manner, weak learners are invested in separating classes whose discrimination is important, at the expense of less relevant classification boundaries. Additional contributions are a family of loss functions along with proof that our algorithm is Boostable in the theoretical sense, as well as an efficient procedure for growing decision trees for use as weak learners. We evaluate our method on a variety of datasets: a collection of synthetic planar data, common UCI datasets, MNIST digits, SUN scenes, and CUB-200 birds. Results show state-of-the-art performance across all datasets against several strong baselines, including non-boosting multi-class approaches.