SPLBoost: An Improved Robust Boosting Algorithm Based on Self-paced Learning

TL;DR

SPLBoost introduces a novel robust boosting algorithm that integrates self-paced learning to enhance noise resistance, offering improved robustness and ease of implementation over traditional boosting methods.

Contribution

The paper proposes SPLBoost, a new boosting algorithm that incorporates self-paced learning to improve robustness against noise and outliers.

Findings

SPLBoost outperforms traditional boosting algorithms in noisy environments.

Theoretical analysis confirms the robustness advantages of SPLBoost.

Experimental results demonstrate the ease of implementation and improved performance.

Abstract

It is known that Boosting can be interpreted as a gradient descent technique to minimize an underlying loss function. Specifically, the underlying loss being minimized by the traditional AdaBoost is the exponential loss, which is proved to be very sensitive to random noise/outliers. Therefore, several Boosting algorithms, e.g., LogitBoost and SavageBoost, have been proposed to improve the robustness of AdaBoost by replacing the exponential loss with some designed robust loss functions. In this work, we present a new way to robustify AdaBoost, i.e., incorporating the robust learning idea of Self-paced Learning (SPL) into Boosting framework. Specifically, we design a new robust Boosting algorithm based on SPL regime, i.e., SPLBoost, which can be easily implemented by slightly modifying off-the-shelf Boosting packages. Extensive experiments and a theoretical characterization are also carried out to illustrate the merits of the proposed SPLBoost.