Online Coordinate Boosting

TL;DR

This paper introduces a new online boosting algorithm that better approximates AdaBoost, with a novel derivation method, and demonstrates its effectiveness through experiments on synthetic and real datasets.

Contribution

A novel online boosting algorithm that closely approximates AdaBoost and a new derivation approach linking previous online boosting methods.

Findings

The new online boosting algorithm more accurately approximates AdaBoost.

The approximation error is comparable to batch AdaBoost on synthetic data.

The method generalizes well on face and MNIST datasets.

Abstract

We present a new online boosting algorithm for adapting the weights of a boosted classifier, which yields a closer approximation to Freund and Schapire's AdaBoost algorithm than previous online boosting algorithms. We also contribute a new way of deriving the online algorithm that ties together previous online boosting work. We assume that the weak hypotheses were selected beforehand, and only their weights are updated during online boosting. The update rule is derived by minimizing AdaBoost's loss when viewed in an incremental form. The equations show that optimization is computationally expensive. However, a fast online approximation is possible. We compare approximation error to batch AdaBoost on synthetic datasets and generalization error on face datasets and the MNIST dataset.