Thompson Sampling-like Algorithms for Stochastic Rising Bandits

TL;DR

This paper explores Thompson sampling-like algorithms for stochastic rising bandits, providing new regret analyses, lower bounds, and empirical comparisons in scenarios where rewards increase over time.

Contribution

It introduces adapted TS algorithms for SRRB, offers novel regret bounds, and compares their performance with existing methods through simulations.

Findings

TS-like algorithms can achieve sublinear regret under certain conditions

A new complexity index influences regret bounds in SRRB

Numerical results show competitive performance of TS-like algorithms

Abstract

Stochastic rising rested bandit (SRRB) is a setting where the arms' expected rewards increase as they are pulled. It models scenarios in which the performances of the different options grow as an effect of an underlying learning process (e.g., online model selection). Even if the bandit literature provides specifically crafted algorithms based on upper-confidence bounds for such a setting, no study about Thompson sampling TS-like algorithms has been performed so far. The strong regularity of the expected rewards in the SRRB setting suggests that specific instances may be tackled effectively using adapted and sliding-window TS approaches. This work provides novel regret analyses for such algorithms in SRRBs, highlighting the challenges and providing new technical tools of independent interest. Our results allow us to identify under which assumptions TS-like algorithms succeed in achieving sublinear regret and which properties of the environment govern the complexity of the regret minimization problem when approached with TS. Furthermore, we provide a regret lower bound based on a complexity index we introduce. Finally, we conduct numerical simulations comparing TS-like algorithms with state-of-the-art approaches for SRRBs in synthetic and real-world settings.