The True Sample Complexity of Identifying Good Arms

TL;DR

This paper redefines sample complexity for identifying good arms in multi-armed bandits, showing it can be much lower than traditional bounds and providing algorithms that match these new bounds.

Contribution

The paper introduces new formal definitions and bounds for multi-armed bandit problems, aligning theoretical results with practical intuition and developing near-optimal algorithms.

Findings

Lower bounds match the intuitive sample complexities of rac{n}{m} and rac{n}{m}k

Practical algorithms achieve nearly matching upper bounds

Sample complexity can be significantly lower than rac{n}{m} in traditional bounds

Abstract

We consider two multi-armed bandit problems with $n$ arms: (i) given an $\epsilon > 0$, identify an arm with mean that is within $\epsilon$ of the largest mean and (ii) given a threshold $\mu_0$ and integer $k$, identify $k$ arms with means larger than $\mu_0$. Existing lower bounds and algorithms for the PAC framework suggest that both of these problems require $\Omega(n)$ samples. However, we argue that these definitions not only conflict with how these algorithms are used in practice, but also that these results disagree with intuition that says (i) requires only $\Theta(\frac{n}{m})$ samples where $m = |\{ i : \mu_i > \max_{i \in [n]} \mu_i - \epsilon\}|$ and (ii) requires $\Theta(\frac{n}{m}k)$ samples where $m = |\{ i : \mu_i > \mu_0 \}|$. We provide definitions that formalize these intuitions, obtain lower bounds that match the above sample complexities, and develop explicit, practical algorithms that achieve nearly matching upper bounds.