Recovering Bandits
Ciara Pike-Burke, Steffen Gr\"unew\"alder

TL;DR
This paper introduces the recovering bandits problem, a dynamic variant of multi-armed bandits where rewards depend on time since last played, and proposes Gaussian process-based methods with regret analysis and empirical validation.
Contribution
It presents a novel formulation of recovering bandits, develops Gaussian process-based algorithms, and provides regret bounds and empirical results for this new problem setting.
Findings
Gaussian process methods effectively estimate rewards over time.
Proposed algorithms achieve sublinear regret bounds.
Empirical studies demonstrate practical performance improvements.
Abstract
We study the recovering bandits problem, a variant of the stochastic multi-armed bandit problem where the expected reward of each arm varies according to some unknown function of the time since the arm was last played. While being a natural extension of the classical bandit problem that arises in many real-world settings, this variation is accompanied by significant difficulties. In particular, methods need to plan ahead and estimate many more quantities than in the classical bandit setting. In this work, we explore the use of Gaussian processes to tackle the estimation and planing problem. We also discuss different regret definitions that let us quantify the performance of the methods. To improve computational efficiency of the methods, we provide an optimistic planning approximation. We complement these discussions with regret bounds and empirical studies.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Optimization and Search Problems · Age of Information Optimization
