Risk-Constrained Thompson Sampling for CVaR Bandits

TL;DR

This paper introduces a risk-aware Thompson Sampling algorithm for multi-armed bandits that optimizes for Conditional Value at Risk (CVaR), demonstrating improved theoretical regret bounds and empirical performance over existing methods.

Contribution

It proposes CVaR-TS, a novel Thompson Sampling-based algorithm tailored for CVaR, with comprehensive theoretical analysis and empirical validation showing superior results.

Findings

CVaR-TS outperforms L/UCB-based algorithms in regret bounds.

Numerical simulations confirm improved empirical performance.

The approach effectively incorporates risk into bandit decision-making.

Abstract

The multi-armed bandit (MAB) problem is a ubiquitous decision-making problem that exemplifies the exploration-exploitation tradeoff. Standard formulations exclude risk in decision making. Risk notably complicates the basic reward-maximising objective, in part because there is no universally agreed definition of it. In this paper, we consider a popular risk measure in quantitative finance known as the Conditional Value at Risk (CVaR). We explore the performance of a Thompson Sampling-based algorithm CVaR-TS under this risk measure. We provide comprehensive comparisons between our regret bounds with state-of-the-art L/UCB-based algorithms in comparable settings and demonstrate their clear improvement in performance. We also include numerical simulations to empirically verify that CVaR-TS outperforms other L/UCB-based algorithms.