Multi-Agent Best Arm Identification in Stochastic Linear Bandits

TL;DR

This paper introduces algorithms for collaborative best-arm identification in stochastic linear bandits with multiple agents, achieving exponential error decay and validated through theoretical analysis and experiments.

Contribution

It proposes two novel algorithms for multi-agent linear bandit best-arm identification applicable to star and arbitrary network topologies, utilizing G-optimal design and communication strategies.

Findings

Algorithms achieve exponentially decaying error probability.

Theoretical guarantees are provided for the proposed methods.

Experimental results outperform existing multi-agent algorithms.

Abstract

We study the problem of collaborative best-arm identification in stochastic linear bandits under a fixed-budget scenario. In our learning model, we first consider multiple agents connected through a star network, interacting with a linear bandit instance in parallel. We then extend our analysis to arbitrary network topologies. The objective of the agents is to collaboratively identify the best arm of the given bandit instance with the help of a central server while minimizing the probability of error in best arm estimation. To this end, we propose two algorithms, MaLinBAI-Star and MaLinBAI-Gen for star networks and networks with arbitrary structure, respectively. Both algorithms utilize the technique of G-optimal design along with the successive elimination based strategy where agents share their knowledge through a central server at each communication round. We demonstrate, both theoretically and empirically, that our algorithms achieve exponentially decaying probability of error in the allocated time budget. Furthermore, experimental results on both synthetic and real-world data validate the effectiveness of our algorithms over the state-of-the art existing multi-agent algorithms.