Efficient Algorithms for Extreme Bandits

TL;DR

This paper introduces efficient algorithms for the Extreme Bandit problem, focusing on maximizing rewards by analyzing the maximum of reward distributions and proposing adaptive strategies with improved empirical and computational performance.

Contribution

The paper develops the QoMax-ETC and QoMax-SDA algorithms, providing novel theoretical analysis and practical methods for the Extreme Bandit problem with enhanced efficiency.

Findings

QoMax-ETC achieves strong asymptotic guarantees.

QoMax-SDA combines adaptivity with subsampling for improved performance.

Algorithms outperform existing methods in empirical tests and resource efficiency.

Abstract

In this paper, we contribute to the Extreme Bandit problem, a variant of Multi-Armed Bandits in which the learner seeks to collect the largest possible reward. We first study the concentration of the maximum of i.i.d random variables under mild assumptions on the tail of the rewards distributions. This analysis motivates the introduction of Quantile of Maxima (QoMax). The properties of QoMax are sufficient to build an Explore-Then-Commit (ETC) strategy, QoMax-ETC, achieving strong asymptotic guarantees despite its simplicity. We then propose and analyze a more adaptive, anytime algorithm, QoMax-SDA, which combines QoMax with a subsampling method recently introduced by Baudry et al. (2021). Both algorithms are more efficient than existing approaches in two aspects (1) they lead to better empirical performance (2) they enjoy a significant reduction of the memory and time complexities.