DCMAC: Demand-aware Customized Multi-Agent Communication via Upper Bound Training

TL;DR

This paper introduces DCMAC, a demand-aware multi-agent communication protocol that uses upper bound training and cross-attention to optimize message exchange, reducing overhead and improving performance in collaborative reinforcement learning.

Contribution

The paper presents a novel demand-aware communication method with upper bound training, enabling agents to generate customized messages based on demand correlation, improving efficiency and training speed.

Findings

DCMAC outperforms baseline algorithms in various scenarios.

It effectively adapts to communication resource constraints.

Accelerates training by utilizing joint observation policies.

Abstract

Efficient communication can enhance the overall performance of collaborative multi-agent reinforcement learning. A common approach is to share observations through full communication, leading to significant communication overhead. Existing work attempts to perceive the global state by conducting teammate model based on local information. However, they ignore that the uncertainty generated by prediction may lead to difficult training. To address this problem, we propose a Demand-aware Customized Multi-Agent Communication (DCMAC) protocol, which use an upper bound training to obtain the ideal policy. By utilizing the demand parsing module, agent can interpret the gain of sending local message on teammate, and generate customized messages via compute the correlation between demands and local observation using cross-attention mechanism. Moreover, our method can adapt to the communication resources of agents and accelerate the training progress by appropriating the ideal policy which is trained with joint observation. Experimental results reveal that DCMAC significantly outperforms the baseline algorithms in both unconstrained and communication constrained scenarios.