Finding All {\epsilon}-Good Arms in Stochastic Bandits

TL;DR

This paper introduces algorithms for identifying all psilon-good arms in stochastic bandits, addressing a previously overlooked problem with significant practical applications and demonstrating strong empirical results.

Contribution

The paper presents the first algorithms specifically designed for all-psilon-good arm identification in stochastic bandits, a new challenge in pure-exploration.

Findings

Algorithms outperform baselines on large-scale datasets

Effective in applications like drug screening and content rating

Addresses a novel problem in bandit theory

Abstract

The pure-exploration problem in stochastic multi-armed bandits aims to find one or more arms with the largest (or near largest) means. Examples include finding an {\epsilon}-good arm, best-arm identification, top-k arm identification, and finding all arms with means above a specified threshold. However, the problem of finding all {\epsilon}-good arms has been overlooked in past work, although arguably this may be the most natural objective in many applications. For example, a virologist may conduct preliminary laboratory experiments on a large candidate set of treatments and move all {\epsilon}-good treatments into more expensive clinical trials. Since the ultimate clinical efficacy is uncertain, it is important to identify all {\epsilon}-good candidates. Mathematically, the all-{\epsilon}-good arm identification problem presents significant new challenges and surprises that do not arise in the pure-exploration objectives studied in the past. We introduce two algorithms to overcome these and demonstrate their great empirical performance on a large-scale crowd-sourced dataset of 2.2M ratings collected by the New Yorker Caption Contest as well as a dataset testing hundreds of possible cancer drugs.