On Characterizing Learnability for Adversarial Noisy Bandits

TL;DR

This paper characterizes the conditions under which adversarial noisy bandit problems are learnable, using a convexified generalized maximin volume, and explores its implications for different types of adversaries and arm spaces.

Contribution

It introduces a new characterization of learnability in adversarial noisy bandits based on a convexified generalized maximin volume, extending understanding to adaptive adversaries and uncountable arm spaces.

Findings

Characterizes learnability for oblivious adversaries using convexified generalized maximin volume.

Shows the same quantity characterizes learnability for adaptive adversaries with countable arm spaces.

Proposes the distribution covering number as a potential measure for uncountable arm spaces.

Abstract

We study adversarial noisy bandits given a known function class $\mathcal{F}$. In each round, the adversary selects a function $f \in \mathcal{F}$, the learner chooses an arm, and then observes a noisy reward determined by the chosen arm and the function $f$. The goal is to minimize the cumulative regret $R(T)$, defined as the difference between the learner's performance and that of the best fixed arm in hindsight over $T$ rounds. We say that a function class $\mathcal{F}$ is learnable if there exists an algorithm achieving sublinear regret. Our main results concern characterizing learnability. The main quantity appearing in our characterization is a convexified variant of the generalized maximin volume introduced by Hanneke and Wang (2025). For oblivious adversaries, we characterize learnability in terms of this convexified generalized maximin volume. For adaptive adversaries, we show that the same quantity characterizes learnability when the arm space is countable. Our analysis builds on a connection between convexified generalized maximin volume and the existence of simple hitting sets. We further conjecture that the same quantity also characterizes learnability when the arm space is uncountable, via its relation to a new complexity measure, which we call the distribution covering number. This notion can be viewed as a strengthened form of the hitting set that still admits efficient learning via the multiplicative weights algorithm. We also pose a number of relevant open questions regarding this problem.