Thompson Sampling for Gaussian Entropic Risk Bandits

TL;DR

This paper investigates a Thompson sampling algorithm for multi-armed bandits that incorporates an entropic risk measure, providing theoretical regret bounds and addressing the challenge of risk-aware decision making.

Contribution

It introduces ERTS, a Thompson sampling-based algorithm for Gaussian entropic risk bandits, with new regret bounds and analysis under this risk measure.

Findings

ERTS achieves sublinear regret bounds.

Theoretical lower bounds are established for the problem.

Risk-aware bandit strategies can be effectively analyzed with this approach.

Abstract

The multi-armed bandit (MAB) problem is a ubiquitous decision-making problem that exemplifies exploration-exploitation tradeoff. Standard formulations exclude risk in decision making. Risknotably complicates the basic reward-maximising objectives, in part because there is no universally agreed definition of it. In this paper, we consider an entropic risk (ER) measure and explore the performance of a Thompson sampling-based algorithm ERTS under this risk measure by providing regret bounds for ERTS and corresponding instance dependent lower bounds.