Efficient Learning of Ensembles with QuadBoost

TL;DR

This paper introduces QuadBoost, a new boosting algorithm supported by a theoretical risk bound, which demonstrates faster empirical error reduction and improved efficiency in ensemble learning compared to AdaBoost.

Contribution

The paper presents QuadBoost, a novel boosting method with simple weight assignment rules, backed by a new risk bound and superior empirical performance.

Findings

QuadBoost has a faster rate of empirical error decrease than AdaBoost.

Experimental results confirm QuadBoost's efficiency in ensemble learning.

A general risk bound for ensembles based on the Lp norm of voter combinations.

Abstract

We first present a general risk bound for ensembles that depends on the Lp norm of the weighted combination of voters which can be selected from a continuous set. We then propose a boosting method, called QuadBoost, which is strongly supported by the general risk bound and has very simple rules for assigning the voters' weights. Moreover, QuadBoost exhibits a rate of decrease of its empirical error which is slightly faster than the one achieved by AdaBoost. The experimental results confirm the expectation of the theory that QuadBoost is a very efficient method for learning ensembles.