Communication Methods in Multi-Agent Reinforcement Learning

TL;DR

This paper reviews various communication techniques in multi-agent reinforcement learning, analyzing their strengths and weaknesses, and emphasizes the importance of problem-specific choices and scalability considerations.

Contribution

It provides an in-depth analysis of 29 publications on communication methods, highlighting the lack of a universal framework and discussing research gaps for real-world applications.

Findings

No single communication method is optimal for all problems.

Scalability depends on low-overhead communication techniques.

Standardized benchmarks and robustness are key research needs.

Abstract

Multi-agent reinforcement learning is a promising research area that extends established reinforcement learning approaches to problems formulated as multi-agent systems. Recently, a multitude of communication methods have been introduced to this field to address problems such as partially observable environments, non-stationarity, and exponentially growing action spaces. Communication further enables efficient cooperation among all agents interacting in an environment. This work aims at providing an overview of communication techniques in multi-agent reinforcement learning. By an in-depth analysis of 29 publications on this topic, the strengths and weaknesses of explicit, implicit, attention-based, graph-based, and hierarchical/role-based communication are evaluated. The results of this comparison show that there is no general, optimal communication framework for every problem. On the contrary, the choice of communication depends heavily on the problem at hand. The comparison also highlights the importance of communication methods with low computational overhead to enable scalability to environments where many agents interact. Finally, the paper discusses current research gaps, emphasizing the need for standardized benchmarking of system-level metrics and improved robustness under realistic communication conditions to enhance the real-world applicability of these approaches.