An Online Boosting Algorithm with Theoretical Justifications

TL;DR

This paper introduces a new online boosting algorithm with solid theoretical guarantees, adapting from offline methods, and demonstrates its effectiveness through experiments on real data.

Contribution

It proposes a novel online boosting algorithm with theoretical justifications, including a new assumption for online weak learners and methods for selecting the number of weak learners.

Findings

The algorithm has strong theoretical guarantees.

It performs favorably compared to existing online boosting methods.

The approach effectively adapts offline boosting techniques to the online setting.

Abstract

We study the task of online boosting--combining online weak learners into an online strong learner. While batch boosting has a sound theoretical foundation, online boosting deserves more study from the theoretical perspective. In this paper, we carefully compare the differences between online and batch boosting, and propose a novel and reasonable assumption for the online weak learner. Based on the assumption, we design an online boosting algorithm with a strong theoretical guarantee by adapting from the offline SmoothBoost algorithm that matches the assumption closely. We further tackle the task of deciding the number of weak learners using established theoretical results for online convex programming and predicting with expert advice. Experiments on real-world data sets demonstrate that the proposed algorithm compares favorably with existing online boosting algorithms.