Safe Linear Thompson Sampling with Side Information

TL;DR

This paper introduces a safe linear Thompson Sampling algorithm for stochastic bandits with linear safety constraints, achieving competitive regret bounds and outperforming UCB-based methods by leveraging the randomness of TS.

Contribution

The paper proposes a novel safe linear Thompson Sampling algorithm with theoretical regret guarantees under safety constraints, extending bandit algorithms to safer decision-making scenarios.

Findings

Achieves regret of order $ ilde{O}(d^{3/2} T^{1/2})$ with safety constraints.

Outperforms UCB-based safe algorithms in expanding safe action sets.

Randomized TS approach enhances safety and exploration in constrained bandits.

Abstract

The design and performance analysis of bandit algorithms in the presence of stage-wise safety or reliability constraints has recently garnered significant interest. In this work, we consider the linear stochastic bandit problem under additional \textit{linear safety constraints} that need to be satisfied at each round. We provide a new safe algorithm based on linear Thompson Sampling (TS) for this problem and show a frequentist regret of order $\mathcal{O} (d^{3/2}\log^{1/2}d \cdot T^{1/2}\log^{3/2}T)$, which remarkably matches the results provided by (Abeille et al., 2017) for the standard linear TS algorithm in the absence of safety constraints. We compare the performance of our algorithm with UCB-based safe algorithms and highlight how the inherently randomized nature of TS leads to a superior performance in expanding the set of safe actions the algorithm has access to at each round.