Best Arm Identification for Stochastic Rising Bandits

TL;DR

This paper studies the fixed-budget best arm identification problem in stochastic rising bandits, proposing algorithms with theoretical guarantees and validating their performance through experiments.

Contribution

It introduces two algorithms for fixed-budget BAI in SRBs, providing theoretical guarantees and matching lower bounds, a novel focus in this setting.

Findings

Algorithms achieve high probability of correct identification with large budgets.

Proved lower bounds match the performance of R-SR algorithm.

Validated effectiveness through experiments on synthetic and real data.

Abstract

Stochastic Rising Bandits (SRBs) model sequential decision-making problems in which the expected reward of the available options increases every time they are selected. This setting captures a wide range of scenarios in which the available options are learning entities whose performance improves (in expectation) over time (e.g., online best model selection). While previous works addressed the regret minimization problem, this paper focuses on the fixed-budget Best Arm Identification (BAI) problem for SRBs. In this scenario, given a fixed budget of rounds, we are asked to provide a recommendation about the best option at the end of the identification process. We propose two algorithms to tackle the above-mentioned setting, namely R-UCBE, which resorts to a UCB-like approach, and R-SR, which employs a successive reject procedure. Then, we prove that, with a sufficiently large budget, they provide guarantees on the probability of properly identifying the optimal option at the end of the learning process and on the simple regret. Furthermore, we derive a lower bound on the error probability, matched by our R-SR (up to constants), and illustrate how the need for a sufficiently large budget is unavoidable in the SRB setting. Finally, we numerically validate the proposed algorithms in both synthetic and realistic environments.