Achieving adaptivity and optimality for multi-armed bandits using Exponential-Kullback Leibler Maillard Sampling

TL;DR

This paper introduces Exp-KL-MS, a new algorithm for multi-armed bandits with exponential rewards, that simultaneously achieves multiple optimality criteria such as asymptotic, minimax, and variance-adaptive bounds.

Contribution

The paper proposes Exp-KL-MS, the first algorithm to satisfy multiple optimality criteria for exponential bandits simultaneously.

Findings

Achieves asymptotic optimality.

Ensures minimax optimality with a sqrt(ln K) factor.

Provides variance-adaptive worst-case regret bounds.

Abstract

We study the problem of $K$-armed bandits with reward distributions belonging to a one-parameter exponential distribution family. In the literature, several criteria have been proposed to evaluate the performance of such algorithms, including Asymptotic Optimality, Minimax Optimality, Sub-UCB, and variance-adaptive worst-case regret bound. Thompson Sampling-based and Upper Confidence Bound-based algorithms have been employed to achieve some of these criteria. However, none of these algorithms simultaneously satisfy all the aforementioned criteria. In this paper, we design an algorithm, Exponential Kullback-Leibler Maillard Sampling (abbrev. Exp-KL-MS), that can achieve multiple optimality criteria simultaneously, including Asymptotic Optimality, Minimax Optimality with a $\sqrt{\ln (K)}$ factor, Sub-UCB, and variance-adaptive worst-case regret bound.