Clus-UCB: A Near-Optimal Algorithm for Clustered Bandits

TL;DR

Clus-UCB is a new algorithm for clustered bandits that leverages known clustering to improve regret bounds and shares information among arms within clusters, outperforming traditional algorithms.

Contribution

The paper introduces Clus-UCB, an algorithm that exploits known clustering to achieve near-optimal regret in stochastic bandits with dependent arms.

Findings

Clus-UCB asymptotically matches the lower regret bound.

Clus-UCB outperforms KL-UCB in simulations.

Sharing information within clusters improves performance.

Abstract

We study a stochastic multi-armed bandit setting where arms are partitioned into known clusters, such that the mean rewards of arms within a cluster differ by at most a known threshold. While the clustering structure is known a priori, the arm means are unknown. We derive an asymptotic lower bound on the regret that improves upon the classical bound of Lai & Robbins (1985). We then propose Clus-UCB, an efficient algorithm that closely matches this lower bound asymptotically. Clus-UCB is designed to exploit the clustering structure and introduces a new index to evaluate an arm, which depends on other arms within the cluster. In this way, arms share information among each other. We present simulation results of our algorithm and compare its performance against KL-UCB and other wellknown algorithms for bandits with dependent arms. Finally, we address some limitations of this work and conclude by mentioning some possible future research.