A Decentralized Communication Framework based on Dual-Level Recurrence for Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a dual-level recurrent communication framework for multi-agent reinforcement learning, enabling decentralized agents to share perceptions fairly and adaptively by separating communication from memory.

Contribution

The paper presents a novel dual-level recurrent model that separates communication and memory, improving fairness and adaptability in decentralized multi-agent systems.

Findings

Outperforms existing decentralized communication frameworks

Effective in both partially and fully observable environments

Enhances fairness and adaptability among agents

Abstract

We propose a model enabling decentralized multiple agents to share their perception of environment in a fair and adaptive way. In our model, both the current message and historical observation are taken into account, and they are handled in the same recurrent model but in different forms. We present a dual-level recurrent communication framework for multi-agent systems, in which the first recurrence occurs in the communication sequence and is used to transmit communication data among agents, while the second recurrence is based on the time sequence and combines the historical observations for each agent. The developed communication flow separates communication messages from memories but allows agents to share their historical observations by the dual-level recurrence. This design makes agents adapt to changeable communication objects, while the communication results are fair to these agents. We provide a sufficient discussion about our method in both partially observable and fully observable environments. The results of several experiments suggest our method outperforms the existing decentralized communication frameworks and the corresponding centralized training method.