Thompson Sampling for Unimodal Bandits

TL;DR

This paper introduces a Thompson Sampling algorithm tailored for unimodal bandits, leveraging the unimodal structure to achieve asymptotic optimality and improved regret bounds over standard methods.

Contribution

It proposes a novel Thompson Sampling approach that exploits unimodal structure, providing theoretical regret bounds and demonstrating superior empirical performance.

Findings

Achieves asymptotically optimal regret for Bernoulli rewards.

Attains logarithmic regret for Gaussian rewards, outperforming standard algorithms.

Shows effectiveness on synthetic and real-world data.

Abstract

In this paper, we propose a Thompson Sampling algorithm for \emph{unimodal} bandits, where the expected reward is unimodal over the partially ordered arms. To exploit the unimodal structure better, at each step, instead of exploration from the entire decision space, our algorithm makes decision according to posterior distribution only in the neighborhood of the arm that has the highest empirical mean estimate. We theoretically prove that, for Bernoulli rewards, the regret of our algorithm reaches the lower bound of unimodal bandits, thus it is asymptotically optimal. For Gaussian rewards, the regret of our algorithm is $\mathcal{O}(\log T)$, which is far better than standard Thompson Sampling algorithms. Extensive experiments demonstrate the effectiveness of the proposed algorithm on both synthetic data sets and the real-world applications.