On Distributed Cooperative Decision-Making in Multiarmed Bandits

TL;DR

This paper introduces a cooperative UCB algorithm for distributed multiarmed bandit problems, combining consensus-based reward estimation and UCB arm selection, with analysis of how communication graph structure affects performance.

Contribution

The paper proposes a novel cooperative UCB algorithm for distributed MABs, integrating consensus and UCB heuristics, and analyzes the impact of network topology on decision accuracy.

Findings

Performance depends on communication graph structure

The cooperative UCB algorithm effectively balances exploration and exploitation

Theoretical analysis characterizes the influence of network topology

Abstract

We study the explore-exploit tradeoff in distributed cooperative decision-making using the context of the multiarmed bandit (MAB) problem. For the distributed cooperative MAB problem, we design the cooperative UCB algorithm that comprises two interleaved distributed processes: (i) running consensus algorithms for estimation of rewards, and (ii) upper-confidence-bound-based heuristics for selection of arms. We rigorously analyze the performance of the cooperative UCB algorithm and characterize the influence of communication graph structure on the decision-making performance of the group.