Improved Algorithms for Adversarial Bandits with Unbounded Losses

TL;DR

This paper introduces two new algorithms, UMAB-NN and UMAB-G, for adversarial multi-armed bandit problems with unbounded losses, achieving adaptive regret bounds and outperforming existing methods.

Contribution

The paper proposes the first adaptive, scale-free algorithms for unbounded losses in adversarial bandits, with theoretical analysis and empirical validation.

Findings

UMAB-NN achieves the first adaptive, scale-free regret bound for non-negative unbounded losses.

UMAB-G can learn from arbitrary unbounded losses, handling both positive and negative values.

The algorithms outperform existing methods in empirical evaluations.

Abstract

We consider the Adversarial Multi-Armed Bandits (MAB) problem with unbounded losses, where the algorithms have no prior knowledge on the sizes of the losses. We present UMAB-NN and UMAB-G, two algorithms for non-negative and general unbounded loss respectively. For non-negative unbounded loss, UMAB-NN achieves the first adaptive and scale free regret bound without uniform exploration. Built up on that, we further develop UMAB-G that can learn from arbitrary unbounded loss. Our analysis reveals the asymmetry between positive and negative losses in the MAB problem and provide additional insights. We also accompany our theoretical findings with extensive empirical evaluations, showing that our algorithms consistently out-performs all existing algorithms that handles unbounded losses.