QuACK: A Multipurpose Queuing Algorithm for Cooperative $k$-Armed Bandits

TL;DR

This paper introduces QuACK, a versatile queuing algorithm that enables any single-agent bandit algorithm to be effectively extended to multi-agent cooperative settings, maintaining regret guarantees and broad applicability.

Contribution

The paper presents a black-box reduction method that generalizes single-agent bandit algorithms to multi-agent cooperative environments with theoretical regret guarantees.

Findings

The reduction preserves regret bounds in subgaussian environments.

The approach is competitive with specialized multi-agent algorithms.

Applicable to various bandit settings like heavy-tailed and duelling bandits.

Abstract

We study the cooperative stochastic $k$-armed bandit problem, where a network of $m$ agents collaborate to find the optimal action. In contrast to most prior work on this problem, which focuses on extending a specific algorithm to the multi-agent setting, we provide a black-box reduction that allows us to extend any single-agent bandit algorithm to the multi-agent setting. Under mild assumptions on the bandit environment, we prove that our reduction transfers the regret guarantees of the single-agent algorithm to the multi-agent setting. These guarantees are tight in subgaussian environments, in that using a near minimax optimal single-player algorithm is near minimax optimal in the multi-player setting up to an additive graph-dependent quantity. Our reduction and theoretical results are also general, and apply to many different bandit settings. By plugging in appropriate single-player algorithms, we can easily develop provably efficient algorithms for many multi-player settings such as heavy-tailed bandits, duelling bandits and bandits with local differential privacy, among others. Experimentally, our approach is competitive with or outperforms specialised multi-agent algorithms.