Individual Regret in Cooperative Stochastic Multi-Armed Bandits

TL;DR

This paper analyzes individual regret bounds in cooperative stochastic multi-armed bandits with multiple agents communicating over a network, introducing a new algorithm with bounds independent of network diameter.

Contribution

It presents the COOP-SE algorithm with regret bounds independent of the communication graph's diameter, and extends analysis to message size and communication rounds.

Findings

Individual regret bound of $O(R/ m + A^2 + A \sqrt{\log T})$

Regret bounds hold with logarithmic message size

Achieves regret of $O(R / m+A \log T)$ with logarithmic communication rounds

Abstract

We study the regret in stochastic Multi-Armed Bandits (MAB) with multiple agents that communicate over an arbitrary connected communication graph. We analyzed a variant of Cooperative Successive Elimination algorithm, COOP-SE, and show an individual regret bound of $O(R/ m + A^2 + A \sqrt{\log T})$ and a nearly matching lower bound. Here $A$ is the number of actions, $T$ the time horizon, $m$ the number of agents, and $R = \sum_{\Delta_i > 0}\log(T)/\Delta_i$ is the optimal single agent regret, where $\Delta_i$ is the sub-optimality gap of action $i$. Our work is the first to show an individual regret bound in cooperative stochastic MAB that is independent of the graph's diameter. When considering communication networks there are additional considerations beyond regret, such as message size and number of communication rounds. First, we show that our regret bound holds even if we restrict the messages to be of logarithmic size. Second, for logarithmic number of communication rounds, we obtain a regret bound of $O(R / m+A \log T)$.