Risk-Aversion in Multi-armed Bandits

TL;DR

This paper introduces a risk-averse multi-armed bandit setting focusing on optimizing risk-return trade-offs rather than expected reward, proposing new algorithms and analyzing their theoretical and empirical performance.

Contribution

It presents a novel risk-averse bandit framework, develops two algorithms tailored for variance-based risk, and provides theoretical guarantees along with preliminary empirical results.

Findings

New risk-averse bandit setting based on variance

Two algorithms with theoretical guarantees

Preliminary empirical results demonstrating effectiveness

Abstract

Stochastic multi-armed bandits solve the Exploration-Exploitation dilemma and ultimately maximize the expected reward. Nonetheless, in many practical problems, maximizing the expected reward is not the most desirable objective. In this paper, we introduce a novel setting based on the principle of risk-aversion where the objective is to compete against the arm with the best risk-return trade-off. This setting proves to be intrinsically more difficult than the standard multi-arm bandit setting due in part to an exploration risk which introduces a regret associated to the variability of an algorithm. Using variance as a measure of risk, we introduce two new algorithms, investigate their theoretical guarantees, and report preliminary empirical results.