A scalable stage-wise approach to large-margin multi-class loss based boosting

TL;DR

This paper introduces a scalable, stage-wise multi-class boosting method that directly maximizes the multi-class margin, significantly improving training efficiency and accuracy for large-scale multi-class classification tasks.

Contribution

The paper proposes a simple, computationally efficient stage-wise boosting approach that maintains margin maximization, outperforming previous methods in speed and accuracy.

Findings

Speeds up training by over 100 times without accuracy loss

Achieves better convergence and generalization in multi-class tasks

Demonstrates effectiveness on machine learning and vision benchmarks

Abstract

We present a scalable and effective classification model to train multi-class boosting for multi-class classification problems. Shen and Hao introduced a direct formulation of multi- class boosting in the sense that it directly maximizes the multi- class margin [C. Shen and Z. Hao, "A direct formulation for totally-corrective multi- class boosting", in Proc. IEEE Conf. Comp. Vis. Patt. Recogn., 2011]. The major problem of their approach is its high computational complexity for training, which hampers its application on real-world problems. In this work, we propose a scalable and simple stage-wise multi-class boosting method, which also directly maximizes the multi-class margin. Our approach of- fers a few advantages: 1) it is simple and computationally efficient to train. The approach can speed up the training time by more than two orders of magnitude without sacrificing the classification accuracy. 2) Like traditional AdaBoost, it is less sensitive to the choice of parameters and empirically demonstrates excellent generalization performance. Experimental results on challenging multi-class machine learning and vision tasks demonstrate that the proposed approach substantially improves the convergence rate and accuracy of the final visual detector at no additional computational cost compared to existing multi-class boosting.