Multi-Agent Reinforcement Learning with Communication-Constrained Priors

TL;DR

This paper introduces a generalized multi-agent reinforcement learning framework that effectively handles lossy communication by distinguishing message types and decoupling their impacts, improving robustness in complex environments.

Contribution

It proposes a novel communication-constrained model and a dual mutual information estimator to enhance multi-agent learning under lossy communication conditions.

Findings

Improved performance on communication-constrained benchmarks

Effective differentiation between lossy and lossless messages

Enhanced robustness in dynamic environments

Abstract

Communication is one of the effective means to improve the learning of cooperative policy in multi-agent systems. However, in most real-world scenarios, lossy communication is a prevalent issue. Existing multi-agent reinforcement learning with communication, due to their limited scalability and robustness, struggles to apply to complex and dynamic real-world environments. To address these challenges, we propose a generalized communication-constrained model to uniformly characterize communication conditions across different scenarios. Based on this, we utilize it as a learning prior to distinguish between lossy and lossless messages for specific scenarios. Additionally, we decouple the impact of lossy and lossless messages on distributed decision-making, drawing on a dual mutual information estimatior, and introduce a communication-constrained multi-agent reinforcement learning framework, quantifying the impact of communication messages into the global reward. Finally, we validate the effectiveness of our approach across several communication-constrained benchmarks.