An Information-Theoretic Analysis for Thompson Sampling with Many Actions

TL;DR

This paper introduces new information-theoretic bounds for Thompson Sampling in multi-action settings, replacing entropy with rate-distortion to better handle many actions, leading to improved regret bounds.

Contribution

It develops novel bounds based on rate-distortion, enabling near-optimal analysis for linear bandits and improved bounds for logistic bandits.

Findings

Recovered near-optimal bounds for linear bandits.

Provided improved regret bounds for logistic bandits.

Bound depends on a computable information-theoretic statistic.

Abstract

Information-theoretic Bayesian regret bounds of Russo and Van Roy capture the dependence of regret on prior uncertainty. However, this dependence is through entropy, which can become arbitrarily large as the number of actions increases. We establish new bounds that depend instead on a notion of rate-distortion. Among other things, this allows us to recover through information-theoretic arguments a near-optimal bound for the linear bandit. We also offer a bound for the logistic bandit that dramatically improves on the best previously available, though this bound depends on an information-theoretic statistic that we have only been able to quantify via computation.