Safe Linear Leveling Bandits

TL;DR

This paper introduces SALE-LTS, a new safe linear bandit algorithm that maintains outcomes near a target level under safety constraints, with proven sublinear regret and promising empirical results.

Contribution

The paper proposes SALE-LTS, a novel acquisition strategy for safe linear bandits focused on leveling, extending traditional reward maximization to safety-critical target level maintenance.

Findings

Achieves sublinear regret comparable to classical bandit algorithms.

Demonstrates effective empirical performance in safety-constrained scenarios.

Introduces a new approach for safety in linear stochastic bandits.

Abstract

Multi-armed bandits (MAB) are extensively studied in various settings where the objective is to \textit{maximize} the actions' outcomes (i.e., rewards) over time. Since safety is crucial in many real-world problems, safe versions of MAB algorithms have also garnered considerable interest. In this work, we tackle a different critical task through the lens of \textit{linear stochastic bandits}, where the aim is to keep the actions' outcomes close to a target level while respecting a \textit{two-sided} safety constraint, which we call \textit{leveling}. Such a task is prevalent in numerous domains. Many healthcare problems, for instance, require keeping a physiological variable in a range and preferably close to a target level. The radical change in our objective necessitates a new acquisition strategy, which is at the heart of a MAB algorithm. We propose SALE-LTS: Safe Leveling via Linear Thompson Sampling algorithm, with a novel acquisition strategy to accommodate our task and show that it achieves sublinear regret with the same time and dimension dependence as previous works on the classical reward maximization problem absent any safety constraint. We demonstrate and discuss our algorithm's empirical performance in detail via thorough experiments.