A minimax and asymptotically optimal algorithm for stochastic bandits

TL;DR

The paper introduces the kl-UCB++ algorithm for stochastic bandits, achieving both asymptotic and minimax optimality, merging two key research directions with clear proofs.

Contribution

It presents the first algorithm that is both asymptotically and minimax optimal for stochastic bandits with exponential families.

Findings

Proves kl-UCB++ is asymptotically optimal.

Shows kl-UCB++ is minimax optimal.

Provides simple, clear proofs for both properties.

Abstract

We propose the kl-UCB ++ algorithm for regret minimization in stochastic bandit models with exponential families of distributions. We prove that it is simultaneously asymptotically optimal (in the sense of Lai and Robbins' lower bound) and minimax optimal. This is the first algorithm proved to enjoy these two properties at the same time. This work thus merges two different lines of research with simple and clear proofs.