Detection Augmented Bandit Procedures for Piecewise Stationary MABs: A Modular Approach
Yu-Han Huang, Argyrios Gerogiannis, Subhonmesh Bose, Venugopal V. Veeravalli

TL;DR
This paper introduces a modular framework for detection-augmented bandit algorithms in non-stationary environments, providing improved theoretical bounds and demonstrating practical effectiveness in piecewise stationary multi-armed bandits.
Contribution
It develops a modular design and analysis framework for change detection in bandit algorithms, with new lower bounds and order-optimal procedures for piecewise stationary environments.
Findings
Modular DAB procedures achieve order-optimal regret bounds.
The framework allows unified analysis of various change detectors and bandit algorithms.
Experimental results show competitive regret performance and effective change detection.
Abstract
Conventional Multi-Armed Bandit (MAB) algorithms are designed for stationary environments, where the reward distributions associated with the arms do not change with time. In many applications, however, the environment is more accurately modeled as being non-stationary. In this work, piecewise stationary MAB (PS-MAB) environments are investigated, in which the reward distributions associated with a subset of the arms change at some change-points and remain stationary between change-points. Our focus is on the asymptotic analysis of PS-MABs, for which practical algorithms based on change detection have been previously proposed. Our goal is to modularize the design and analysis of such Detection Augmented Bandit (DAB) procedures. To this end, we first provide novel, improved performance lower bounds for PS-MABs. Then, we identify the requirements for stationary bandit algorithms and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Control Systems Optimization · Fault Detection and Control Systems · Advanced Statistical Process Monitoring
MethodsFocus
