Thompson Sampling for 1-Dimensional Exponential Family Bandits

TL;DR

This paper proves the asymptotic optimality of Thompson Sampling with Jeffreys prior for 1-dimensional exponential family bandits, extending theoretical guarantees beyond Bernoulli cases and covering heavy-tailed distributions.

Contribution

It extends the theoretical analysis of Thompson Sampling to all 1D exponential family bandits, including heavy-tailed distributions, using closed-form KL divergence and Fisher information.

Findings

Proves asymptotic optimality of Thompson Sampling with Jeffreys prior.

Provides finite-time exponential concentration inequalities for exponential families.

Covers distributions without previously proposed optimistic algorithms.

Abstract

Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential family bandits. Our proof builds on previous work, but also makes extensive use of closed forms for Kullback-Leibler divergence and Fisher information (and thus Jeffreys prior) available in an exponential family. This allow us to give a finite time exponential concentration inequality for posterior distributions on exponential families that may be of interest in its own right. Moreover our analysis covers some distributions for which no optimistic algorithm has yet been proposed, including heavy-tailed exponential families.