Bounded regret in stochastic multi-armed bandits

TL;DR

This paper introduces a new randomized policy for stochastic multi-armed bandits that achieves bounded regret over time when the optimal arm's value and a positive gap lower bound are known, with proofs of lower bounds clarifying the limits of bounded regret.

Contribution

The paper presents a novel randomized policy that guarantees bounded regret in the bandit setting under specific knowledge assumptions, advancing understanding of regret bounds.

Findings

The proposed policy achieves bounded regret when the optimal arm's value and a positive gap lower bound are known.

Lower bounds demonstrate that bounded regret is impossible if only the gap or only the optimal value is known.

Bounded regret of order 1/Δ cannot be achieved if only the optimal value is known.

Abstract

We study the stochastic multi-armed bandit problem when one knows the value $\mu^{(\star)}$ of an optimal arm, as a well as a positive lower bound on the smallest positive gap $\Delta$. We propose a new randomized policy that attains a regret {\em uniformly bounded over time} in this setting. We also prove several lower bounds, which show in particular that bounded regret is not possible if one only knows $\Delta$, and bounded regret of order $1/\Delta$ is not possible if one only knows $\mu^{(\star)}$