Multiclass Boosting: Simple and Intuitive Weak Learning Criteria

TL;DR

This paper introduces a simple, efficient multiclass boosting algorithm with weak learning conditions that are intuitive and do not rely on realizability, providing theoretical insights and improved bounds in List PAC learning.

Contribution

It presents a new weak learning criterion for multiclass boosting and a boosting algorithm with bounds independent of class count, along with applications in List PAC learning.

Findings

Sample and oracle complexity bounds are independent of the number of classes.

Established an equivalence to weak PAC learning.

Provided improved error bounds for large list sizes.

Abstract

We study a generalization of boosting to the multiclass setting. We introduce a weak learning condition for multiclass classification that captures the original notion of weak learnability as being "slightly better than random guessing". We give a simple and efficient boosting algorithm, that does not require realizability assumptions and its sample and oracle complexity bounds are independent of the number of classes. In addition, we utilize our new boosting technique in several theoretical applications within the context of List PAC Learning. First, we establish an equivalence to weak PAC learning. Furthermore, we present a new result on boosting for list learners, as well as provide a novel proof for the characterization of multiclass PAC learning and List PAC learning. Notably, our technique gives rise to a simplified analysis, and also implies an improved error bound for large list sizes, compared to previous results.