Generalized Risk-Aversion in Stochastic Multi-Armed Bandits

TL;DR

This paper explores how to minimize regret in stochastic multi-armed bandits when the evaluation criterion is a general function of mean and variance, identifying conditions for learnability and limitations of algorithms.

Contribution

It characterizes conditions for learnability with general risk measures and shows that some functions prevent sublinear regret with natural algorithms.

Findings

Learning is possible under certain conditions on the risk function.

Some risk functions make sublinear regret impossible for natural algorithms.

The paper provides examples illustrating these limitations.

Abstract

We consider the problem of minimizing the regret in stochastic multi-armed bandit, when the measure of goodness of an arm is not the mean return, but some general function of the mean and the variance.We characterize the conditions under which learning is possible and present examples for which no natural algorithm can achieve sublinear regret.