DAL: A Practical Prior-Free Black-Box Framework for Non-Stationary Bandits

TL;DR

DAL is a versatile, prior-free framework that enhances stationary bandit algorithms with change detection to effectively handle non-stationary environments, demonstrated through extensive experiments and theoretical insights.

Contribution

Introduces DAL, a practical black-box framework that augments any stationary bandit algorithm with change detection for non-stationary bandit problems without prior knowledge.

Findings

DAL outperforms state-of-the-art methods in diverse scenarios

It is applicable to all common bandit variants

Demonstrates strong empirical and theoretical performance

Abstract

We introduce a practical, black-box framework termed Detection Augmented Learning (DAL) for the problem of non-stationary bandits without prior knowledge of the underlying non-stationarity. DAL accepts any stationary bandit algorithm as input and augments it with a change detector, enabling applicability to all common bandit variants. Extensive experimentation demonstrates that DAL consistently surpasses current state-of-the-art methods across diverse non-stationary scenarios, including synthetic benchmarks and real-world datasets, underscoring its versatility and scalability. We provide theoretical insights into DAL's strong empirical performance, complemented by thorough experimental validation.