Unifying Clustered and Non-stationary Bandits

TL;DR

This paper unifies the approaches to non-stationary bandits and online clustering of bandits through a test of homogeneity, enabling change detection and cluster identification within a single framework, supported by theoretical and empirical validation.

Contribution

It introduces a unified framework that combines change detection and clustering in bandit problems using a test of homogeneity, advancing the integration of these research areas.

Findings

Rigorous regret bounds established for the unified method.

Extensive experiments show improved adaptability across environments.

Framework effectively detects changes and identifies clusters in real-time.

Abstract

Non-stationary bandits and online clustering of bandits lift the restrictive assumptions in contextual bandits and provide solutions to many important real-world scenarios. Though the essence in solving these two problems overlaps considerably, they have been studied independently. In this paper, we connect these two strands of bandit research under the notion of test of homogeneity, which seamlessly addresses change detection for non-stationary bandit and cluster identification for online clustering of bandit in a unified solution framework. Rigorous regret analysis and extensive empirical evaluations demonstrate the value of our proposed solution, especially its flexibility in handling various environment assumptions.