The Pareto Frontier of Instance-Dependent Guarantees in Multi-Player Multi-Armed Bandits with no Communication

TL;DR

This paper characterizes the fundamental trade-offs in achieving optimal instance-dependent regret in multi-player multi-armed bandits without communication, revealing limitations and proposing a generalized algorithm resilient to adversarial feedback.

Contribution

It provides a complete Pareto frontier of achievable regret guarantees without communication and introduces a new topological lower bound technique.

Findings

Optimal regret guarantees are impossible for some gaps without communication.

The paper characterizes all Pareto optimal trade-offs in regret.

A generalized algorithm achieves these trade-offs even with adversarial feedback.

Abstract

We study the stochastic multi-player multi-armed bandit problem. In this problem, $m$ players cooperate to maximize their total reward from $K > m$ arms. However the players cannot communicate and are penalized (e.g. receive no reward) if they pull the same arm at the same time. We ask whether it is possible to obtain optimal instance-dependent regret $\tilde{O}(1/\Delta)$ where $\Delta$ is the gap between the $m$-th and $m+1$-st best arms. Such guarantees were recently achieved in a model allowing the players to implicitly communicate through intentional collisions. Surprisingly, we show that with no communication at all, such guarantees are not achievable. In fact, obtaining the optimal $\tilde{O}(1/\Delta)$ regret for some values of $\Delta$ necessarily implies strictly sub-optimal regret in other regimes. Our main result is a complete characterization of the Pareto optimal instance-dependent trade-offs that are possible with no communication. Our algorithm generalizes that of Bubeck, Budzinski, and the second author. As there, our algorithm succeeds even when feedback upon collision can be corrupted by an adaptive adversary, thanks to a strong no-collision property. Our lower bound is based on topological obstructions at multiple scales and is completely new.