Order Optimal Regret Bounds for Sharpe Ratio Optimization under Thompson Sampling

TL;DR

This paper introduces a Bayesian Thompson Sampling algorithm for optimizing the Sharpe ratio in multi-armed bandits, achieving order-optimal regret bounds and demonstrating superior performance over existing methods.

Contribution

The paper develops SRTS, a risk-aware Thompson Sampling algorithm with a unified approach for different risk regimes, and provides theoretical guarantees of its optimality.

Findings

Achieves an $ ilde{O}( ext{log } n)$ regret bound for SR optimization.

Provides a matching lower bound, establishing order-optimality.

Shows improved empirical performance over existing risk-aware bandit algorithms.

Abstract

In this paper, we study sequential decision-making for maximizing the Sharpe ratio (SR) in a stochastic multi-armed bandit (MAB) setting. Unlike standard bandit formulations that maximize cumulative reward, SR optimization requires balancing expected return and reward variability. As a result, the learning objective depends jointly on the mean and variance of the reward distribution and takes a fractional form. To address this problem, we propose the Sharpe Ratio Thompson Sampling \texttt{SRTS}, a Bayesian algorithm for risk-adjusted exploration. For Gaussian reward models, the algorithm employs a Normal-Gamma conjugate posterior to capture uncertainty in both the mean and the precision of each arm. In contrast to additive mean-variance (MV) formulations, which often require different algorithms across risk regimes, the fractional SR objective yields a single sampling rule that applies uniformly across risk tolerances. On the theoretical side, we develop a regret decomposition tailored to the SR objective and introduce a decoupling approach that separates the contributions of mean and variance uncertainty. This framework allows us to control the interaction between the Gaussian mean samples and the Gamma precision samples arising in the posterior. Using these results, we establish a finite-time distribution-dependent $\mathcal{O}(\log n)$ upper bound on the expected regret. We further derive a matching information-theoretic lower bound using a change-of-measure argument, showing that the proposed algorithm is order-optimal. Finally, experiments on synthetic bandit environments illustrate the performance of \texttt{SRTS} and demonstrate improvements over existing risk-aware bandit algorithms across a range of risk-return settings.