Learning Efficient Communication Protocols for Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a generalized framework and novel metrics for learning and optimizing efficient multi-round communication protocols in multi-agent reinforcement learning, improving cooperation and reducing redundant messaging.

Contribution

It proposes a new framework with three communication efficiency metrics and integrates them into the learning process to enhance communication effectiveness and efficiency.

Findings

Learned protocols significantly reduce communication volume.

Protocols improve cooperation success rates.

Enhanced efficiency without sacrificing task performance.

Abstract

Multi-Agent Systems (MAS) have emerged as a powerful paradigm for modeling complex interactions among autonomous entities in distributed environments. In Multi-Agent Reinforcement Learning (MARL), communication enables coordination but can lead to inefficient information exchange, since agents may generate redundant or non-essential messages. While prior work has focused on boosting task performance with information exchange, the existing research lacks a thorough investigation of both the appropriate definition and the optimization of communication protocols (communication topology and message). To fill this gap, we introduce a generalized framework for learning multi-round communication protocols that are both effective and efficient. Within this framework, we propose three novel Communication Efficiency Metrics (CEMs) to guide and evaluate the learning process: the Information Entropy Efficiency Index (IEI) and Specialization Efficiency Index (SEI) for efficiency-augmented optimization, and the Topology Efficiency Index (TEI) for explicit evaluation. We integrate IEI and SEI as the adjusted loss functions to promote informative messaging and role specialization, while using TEI to quantify the trade-off between communication volume and task performance. Through comprehensive experiments, we demonstrate that our learned communication protocol can significantly enhance communication efficiency and achieves better cooperation performance with improved success rates.