Optimal $\delta$-Correct Best-Arm Selection for Heavy-Tailed Distributions

TL;DR

This paper develops a $\, ext{delta}$-correct algorithm for best-arm identification in heavy-tailed distributions, matching lower bounds under mild moment restrictions and introducing batch processing for efficiency.

Contribution

It introduces a $\, ext{delta}$-correct algorithm that achieves optimal sample complexity for heavy-tailed distributions with bounded moments, extending previous results beyond exponential families.

Findings

The algorithm matches the asymptotic lower bound as $\, ext{delta}$ approaches zero.

Batch processing significantly speeds up the algorithm.

A mild moment condition suffices for optimality in heavy-tailed settings.

Abstract

Given a finite set of unknown distributions or arms that can be sampled, we consider the problem of identifying the one with the maximum mean using a $\delta$-correct algorithm (an adaptive, sequential algorithm that restricts the probability of error to a specified $\delta$) that has minimum sample complexity. Lower bounds for $\delta$-correct algorithms are well known. $\delta$-correct algorithms that match the lower bound asymptotically as $\delta$ reduces to zero have been previously developed when arm distributions are restricted to a single parameter exponential family. In this paper, we first observe a negative result that some restrictions are essential, as otherwise, under a $\delta$-correct algorithm, distributions with unbounded support would require an infinite number of samples in expectation. We then propose a $\delta$-correct algorithm that matches the lower bound as $\delta$ reduces to zero under the mild restriction that a known bound on the expectation of $(1+\epsilon)^{th}$ moment of the underlying random variables exists, for $\epsilon > 0$. We also propose batch processing and identify near-optimal batch sizes to speed up the proposed algorithm substantially. The best-arm problem has many learning applications, including recommendation systems and product selection. It is also a well-studied classic problem in the simulation community.