Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems

TL;DR

This paper introduces TGCNet, a novel multi-agent reinforcement learning method that uses dynamic directed graphs and Transformer-based communication to improve cooperation in partially observed tasks, achieving state-of-the-art results.

Contribution

The paper proposes TGCNet, a new MARL algorithm that models dynamic directed communication graphs and integrates graph coarsening with Transformer decoders for enhanced cooperation.

Findings

Achieves state-of-the-art performance on multiple MARL benchmarks.

Validates the effectiveness of dynamic directed graph communication.

Demonstrates the benefits of graph coarsening networks in training.

Abstract

Multi-agent systems must learn to communicate and understand interactions between agents to achieve cooperative goals in partially observed tasks. However, existing approaches lack a dynamic directed communication mechanism and rely on global states, thus diminishing the role of communication in centralized training. Thus, we propose the Transformer-based graph coarsening network (TGCNet), a novel multi-agent reinforcement learning (MARL) algorithm. TGCNet learns the topological structure of a dynamic directed graph to represent the communication policy and integrates graph coarsening networks to approximate the representation of global state during training. It also utilizes the Transformer decoder for feature extraction during execution. Experiments on multiple cooperative MARL benchmarks demonstrate state-of-the-art performance compared to popular MARL algorithms. Further ablation studies validate the effectiveness of our dynamic directed graph communication mechanism and graph coarsening networks.