A Finite-Time Analysis of Multi-armed Bandits Problems with Kullback-Leibler Divergences

TL;DR

This paper provides a finite-time analysis of a Kullback-Leibler-based algorithm for stochastic multi-armed bandits with finite support distributions, achieving regret bounds better than previous algorithms.

Contribution

It offers the first finite-time bounds for a KL-based bandit algorithm with finite support distributions, matching asymptotic lower bounds.

Findings

Finite-time regret bounds are tighter than previous UCB-type algorithms.

The algorithm's asymptotic regret matches the known lower bounds.

Results demonstrate improved performance in finite-time scenarios.

Abstract

We consider a Kullback-Leibler-based algorithm for the stochastic multi-armed bandit problem in the case of distributions with finite supports (not necessarily known beforehand), whose asymptotic regret matches the lower bound of \cite{Burnetas96}. Our contribution is to provide a finite-time analysis of this algorithm; we get bounds whose main terms are smaller than the ones of previously known algorithms with finite-time analyses (like UCB-type algorithms).