Near-Optimal Regret for Distributed Adversarial Bandits: A Black-Box Approach

TL;DR

This paper introduces a near-optimal algorithm for distributed adversarial bandits that leverages a black-box reduction to handle delayed feedback, significantly improving regret bounds and extending to linear bandits.

Contribution

The paper presents a novel black-box reduction approach for distributed adversarial bandits, achieving near-optimal regret bounds and extending to linear bandits with minimal communication.

Findings

Achieves minimax regret of rac12;( ho^{-1/2}+K/N)T

Introduces a gossip-based algorithm with improved regret bounds over previous work

Extends framework to distributed linear bandits with low communication cost

Abstract

We study distributed adversarial bandits, where $N$ agents cooperate to minimize the global average loss while observing only their own local losses. We show that the minimax regret for this problem is $\tilde{\Theta}(\sqrt{(\rho^{-1/2}+K/N)T})$, where $T$ is the horizon, $K$ is the number of actions, and $\rho$ is the spectral gap of the communication matrix. Our algorithm, based on a novel black-box reduction to bandits with delayed feedback, requires agents to communicate only through gossip. It achieves an upper bound that significantly improves over the previous best bound $\tilde{O}(\rho^{-1/3}(KT)^{2/3})$ of Yi and Vojnovic (2023). We complement this result with a matching lower bound, showing that the problem's difficulty decomposes into a communication cost $\rho^{-1/4}\sqrt{T}$ and a bandit cost $\sqrt{KT/N}$. We further demonstrate the versatility of our approach by deriving first-order and best-of-both-worlds bounds in the distributed adversarial setting. Finally, we extend our framework to distributed linear bandits in $R^d$, obtaining a regret bound of $\tilde{O}(\sqrt{(\rho^{-1/2}+1/N)dT})$, achieved with only $O(d)$ communication cost per agent and per round via a volumetric spanner.