A Decentralized Communication Policy for Multi Agent Multi Armed Bandit Problems

TL;DR

This paper introduces a decentralized communication policy for multi-agent multi-armed bandit problems, leveraging local sampling and targeted neighbor communication to improve exploration efficiency and scalability.

Contribution

It presents a novel communication strategy based on UCB and selective neighbor communication, outperforming random policies and enhancing scalability in multi-agent MAB settings.

Findings

Significant performance improvement over random communication policies.

Cost-effective and scalable to large agent networks.

Effective exploration and exploitation balance achieved.

Abstract

This paper proposes a novel policy for a group of agents to, individually as well as collectively, solve a multi armed bandit (MAB) problem. The policy relies solely on the information that an agent has obtained through sampling of the options on its own and through communication with neighbors. The option selection policy is based on an Upper Confidence Based (UCB) strategy while the communication strategy that is proposed forces agents to communicate with other agents who they believe are most likely to be exploring than exploiting. The overall strategy is shown to significantly outperform an independent Erd\H{o}s-R\'{e}nyi (ER) graph based random communication policy. The policy is shown to be cost effective in terms of communication and thus to be easily scalable to a large network of agents.