When and why randomised exploration works (in linear bandits)

TL;DR

This paper analyzes when and why randomized exploration algorithms like Thompson sampling are effective in linear bandits, providing new regret bounds without relying on traditional optimism assumptions.

Contribution

It introduces a novel analysis method for randomized exploration in linear bandits, showing Thompson sampling can achieve optimal regret bounds in certain smooth, convex action spaces.

Findings

Thompson sampling achieves $O(d\,\sqrt{n}\log(n))$ regret in smooth, convex action spaces.

First demonstration of optimal dimension dependence for Thompson sampling in non-trivial linear bandit settings.

Analysis does not depend on forced optimism or posterior inflation techniques.

Abstract

We provide an approach for the analysis of randomised exploration algorithms like Thompson sampling that does not rely on forced optimism or posterior inflation. With this, we demonstrate that in the $d$-dimensional linear bandit setting, when the action space is smooth and strongly convex, randomised exploration algorithms enjoy an $n$-step regret bound of the order $O(d\sqrt{n} \log(n))$. Notably, this shows for the first time that there exist non-trivial linear bandit settings where Thompson sampling can achieve optimal dimension dependence in the regret.