Boosting as a Product of Experts

TL;DR

This paper introduces a probabilistic boosting model called POE-Boost, which improves ensemble learning by ensuring likelihood increases with each expert added, and extends it to probabilistic hypotheses with better generalization.

Contribution

It presents a novel probabilistic framework for boosting as a Product of Experts and extends it to probabilistic hypotheses, enhancing generalization performance.

Findings

POE-Boost is similar to AdaBoost under certain conditions.

POEBoost.CS outperforms other algorithms in generalization.

The model ensures likelihood does not decrease with new experts.

Abstract

In this paper, we derive a novel probabilistic model of boosting as a Product of Experts. We re-derive the boosting algorithm as a greedy incremental model selection procedure which ensures that addition of new experts to the ensemble does not decrease the likelihood of the data. These learning rules lead to a generic boosting algorithm - POE- Boost which turns out to be similar to the AdaBoost algorithm under certain assumptions on the expert probabilities. The paper then extends the POEBoost algorithm to POEBoost.CS which handles hypothesis that produce probabilistic predictions. This new algorithm is shown to have better generalization performance compared to other state of the art algorithms.