Non-Convex Boosting Overcomes Random Label Noise

TL;DR

This paper evaluates boosting algorithms' robustness to random label noise, showing BrownBoost and RobustBoost outperform AdaBoost and LogitBoost in noisy conditions, with differences explained by margin distributions.

Contribution

It provides experimental evidence that non-convex boosting algorithms are more robust to label noise than traditional convex boosting methods.

Findings

BrownBoost and RobustBoost outperform AdaBoost and LogitBoost under label noise

The performance difference is linked to margin distribution differences

Non-convex boosting algorithms are more noise-tolerant

Abstract

The sensitivity of Adaboost to random label noise is a well-studied problem. LogitBoost, BrownBoost and RobustBoost are boosting algorithms claimed to be less sensitive to noise than AdaBoost. We present the results of experiments evaluating these algorithms on both synthetic and real datasets. We compare the performance on each of datasets when the labels are corrupted by different levels of independent label noise. In presence of random label noise, we found that BrownBoost and RobustBoost perform significantly better than AdaBoost and LogitBoost, while the difference between each pair of algorithms is insignificant. We provide an explanation for the difference based on the margin distributions of the algorithms.