An Information-Theoretic Analysis of Thompson Sampling with Infinite Action Spaces

TL;DR

This paper extends the information-theoretic analysis of Thompson Sampling to infinite and continuous action spaces, providing new regret bounds that consider the complexity of the action space.

Contribution

It generalizes existing finite-action bounds to infinite and continuous settings, including Lipschitz continuous reward functions.

Findings

Derived regret bounds for infinite action spaces

Extended rate-distortion analysis to continuous actions

Accounted for action space complexity in bounds

Abstract

This paper studies the Bayesian regret of the Thompson Sampling algorithm for bandit problems, building on the information-theoretic framework introduced by Russo and Van Roy (2015). Specifically, it extends the rate-distortion analysis of Dong and Van Roy (2018), which provides near-optimal bounds for linear bandits. A limitation of these results is the assumption of a finite action space. We address this by extending the analysis to settings with infinite and continuous action spaces. Additionally, we specialize our results to bandit problems with expected rewards that are Lipschitz continuous with respect to the action space, deriving a regret bound that explicitly accounts for the complexity of the action space.