Lookahead identification in adversarial bandits: accuracy and memory bounds

TL;DR

This paper introduces lookahead identification in adversarial multi-armed bandits, demonstrating that accurate future arm predictions are possible with specific accuracy bounds and memory requirements, even in challenging environments.

Contribution

It defines the novel task of lookahead identification, analyzes achievable accuracy bounds, and explores memory complexity, including under sparsity conditions.

Findings

Achieves 1/\u221a{\u221a{ ext{log} T}} accuracy over { ext{O}(\u221a{ ext{log} T})} prediction windows.

Proves a lower bound of { ext{1/ ext{log} T}} accuracy, showing the bounds are tight.

Shows nontrivial accuracy requires { ext{} ext{memory} ext{of} ext{} ext{at} ext{} ext{least} ext{} ext{bits}.

Abstract

We study an identification problem in multi-armed bandits. In each round a learner selects one of $K$ arms and observes its reward, with the goal of eventually identifying an arm that will perform best at a {\it future} time. In adversarial environments, however, past performance may offer little information about the future, raising the question of whether meaningful identification is possible at all. In this work, we introduce \emph{lookahead identification}, a task in which the goal of the learner is to select a future prediction window and commit in advance to an arm whose average reward over that window is within $\varepsilon$ of optimal. Our analysis characterizes both the achievable accuracy of lookahead identification and the memory resources required to obtain it. From an accuracy standpoint, for any horizon $T$ we give an algorithm achieving $\varepsilon = O\bigl(1/\sqrt{\log T}\bigr)$ over $\Omega(\sqrt{T})$ prediction windows. This demonstrates that, perhaps surprisingly, identification is possible in adversarial settings, despite significant lack of information. We also prove a near-matching lower bound showing that $\varepsilon = \Omega\bigl(1/\log T\bigr)$ is unavoidable. We then turn to investigate the role of memory in our problem, first proving that any algorithm achieving nontrivial accuracy requires $\Omega(K)$ bits of memory. Under a natural \emph{local sparsity} condition, we show that the same accuracy guarantees can be achieved using only poly-logarithmic memory.