Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost

TL;DR

This paper introduces a robust version of LogitBoost with a stable tree-split criterion and an adaptive base class extension for multi-class classification, demonstrating superior performance over existing methods.

Contribution

It develops a numerically stable formulation of LogitBoost and proposes abc-LogitBoost, combining robustness with adaptive base class strategies for improved multi-class classification.

Findings

abc-LogitBoost outperforms mart, abcmart, and traditional LogitBoost.

The new formulation enhances numerical stability.

Experimental results show superior accuracy in multi-class tasks.

Abstract

Logitboost is an influential boosting algorithm for classification. In this paper, we develop robust logitboost to provide an explicit formulation of tree-split criterion for building weak learners (regression trees) for logitboost. This formulation leads to a numerically stable implementation of logitboost. We then propose abc-logitboost for multi-class classification, by combining robust logitboost with the prior work of abc-boost. Previously, abc-boost was implemented as abc-mart using the mart algorithm. Our extensive experiments on multi-class classification compare four algorithms: mart, abcmart, (robust) logitboost, and abc-logitboost, and demonstrate the superiority of abc-logitboost. Comparisons with other learning methods including SVM and deep learning are also available through prior publications.