Fast Rates for Bandit PAC Multiclass Classification

TL;DR

This paper introduces a new algorithm for multiclass PAC learning with bandit feedback that achieves faster sample complexity rates, matching the full-information case and resolving open questions about the cost of bandit feedback.

Contribution

The paper presents a novel learning algorithm with improved sample complexity bounds for agnostic multiclass PAC learning under bandit feedback, extending to general hypothesis classes and establishing optimal rates.

Findings

Achieves sample complexity of $O(( ext{poly}(K) + 1/ ext{ε}^2) imes ext{log}(|H|/ ext{δ}))$

Matches the optimal rate in the full-information setting for general classes

Shows the bandit feedback cost is only $O(1)$ in the agnostic case as ε approaches zero.

Abstract

We study multiclass PAC learning with bandit feedback, where inputs are classified into one of $K$ possible labels and feedback is limited to whether or not the predicted labels are correct. Our main contribution is in designing a novel learning algorithm for the agnostic $(\varepsilon,\delta)$-PAC version of the problem, with sample complexity of $O\big( (\operatorname{poly}(K) + 1 / \varepsilon^2) \log (|H| / \delta) \big)$ for any finite hypothesis class $H$. In terms of the leading dependence on $\varepsilon$, this improves upon existing bounds for the problem, that are of the form $O(K/\varepsilon^2)$. We also provide an extension of this result to general classes and establish similar sample complexity bounds in which $\log |H|$ is replaced by the Natarajan dimension. This matches the optimal rate in the full-information version of the problem and resolves an open question studied by Daniely, Sabato, Ben-David, and Shalev-Shwartz (2011) who demonstrated that the multiplicative price of bandit feedback in realizable PAC learning is $\Theta(K)$. We complement this by revealing a stark contrast with the agnostic case, where the price of bandit feedback is only $O(1)$ as $\varepsilon \to 0$. Our algorithm utilizes a stochastic optimization technique to minimize a log-barrier potential based on Frank-Wolfe updates for computing a low-variance exploration distribution over the hypotheses, and is made computationally efficient provided access to an ERM oracle over $H$.