Multiclass Online Learnability under Bandit Feedback

TL;DR

This paper characterizes the conditions under which online multiclass classification with bandit feedback is learnable, establishing the Bandit Littlestone dimension as a key measure and highlighting differences from full-information scenarios.

Contribution

It extends the theoretical understanding of bandit online learnability by linking it to the Bandit Littlestone dimension and contrasting it with full-information settings.

Findings

Finiteness of Bandit Littlestone dimension is necessary and sufficient for learnability.

Sequential uniform convergence is necessary but not sufficient for bandit learnability.

Results apply even when the label space is unbounded.

Abstract

We study online multiclass classification under bandit feedback. We extend the results of Daniely and Helbertal [2013] by showing that the finiteness of the Bandit Littlestone dimension is necessary and sufficient for bandit online learnability even when the label space is unbounded. Moreover, we show that, unlike the full-information setting, sequential uniform convergence is necessary but not sufficient for bandit online learnability. Our result complements the recent work by Hanneke, Moran, Raman, Subedi, and Tewari [2023] who show that the Littlestone dimension characterizes online multiclass learnability in the full-information setting even when the label space is unbounded.