Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control

TL;DR

This paper introduces Variance Based Control (VBC), a technique that reduces communication overhead in multi-agent reinforcement learning by limiting message variance, leading to more efficient and effective collaboration.

Contribution

The paper proposes a novel variance-based method to improve communication efficiency in MARL, significantly reducing message exchange while enhancing agent collaboration.

Findings

Achieves 2-10x lower communication overhead compared to state-of-the-art methods.

Enables agents to develop more sophisticated strategies in StarCraft II benchmarks.

Improves overall performance and coordination among agents.

Abstract

Multi-agent reinforcement learning (MARL) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in MARL. In this work, we propose Variance Based Control (VBC), a simple yet efficient technique to improve communication efficiency in MARL. By limiting the variance of the exchanged messages between agents during the training phase, the noisy component in the messages can be eliminated effectively, while the useful part can be preserved and utilized by the agents for better performance. Our evaluation using a challenging set of StarCraft II benchmarks indicates that our method achieves $2-10\times$ lower in communication overhead than state-of-the-art MARL algorithms, while allowing agents to better collaborate by developing sophisticated strategies.