Tracking Most Significant Shifts in Infinite-Armed Bandits
Joe Suk, Jung-hun Kim

TL;DR
This paper develops a parameter-free, optimal regret algorithm for infinite-armed bandits with non-stationary rewards, introducing a blackbox scheme and a notion of significant shifts to adaptively improve performance.
Contribution
It presents the first parameter-free, optimal regret bounds for all regimes of non-stationarity in infinite-armed bandits, relaxing distributional assumptions and introducing a shift-based analysis.
Findings
Achieves parameter-free optimal regret bounds for all non-stationarity regimes.
Introduces a blackbox scheme converting finite-armed algorithms for non-stationary environments.
Shows that rising rewards do not impact the difficulty of non-stationarity.
Abstract
We study an infinite-armed bandit problem where actions' mean rewards are initially sampled from a reservoir distribution. Most prior works in this setting focused on stationary rewards (Berry et al., 1997; Wang et al., 2008; Bonald and Proutiere, 2013; Carpentier and Valko, 2015) with the more challenging adversarial/non-stationary variant only recently studied in the context of rotting/decreasing rewards (Kim et al., 2022; 2024). Furthermore, optimal regret upper bounds were only achieved using parameter knowledge of non-stationarity and only known for certain regimes of regularity of the reservoir. This work shows the first parameter-free optimal regret bounds for all regimes while also relaxing distributional assumptions on the reservoir. We first introduce a blackbox scheme to convert a finite-armed MAB algorithm designed for near-stationary environments into a parameter-free…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Data Stream Mining Techniques
