A Survey of Multi-Agent Deep Reinforcement Learning with Graph Neural Network-Based Communication

TL;DR

This paper surveys recent multi-agent reinforcement learning approaches that utilize graph neural networks for communication, aiming to clarify their underlying structures and frameworks.

Contribution

It introduces a generalized GNN-based communication process to better classify and understand MARL methods with GNNs.

Findings

GNN-based communication enhances agent coordination in MARL.

The survey clarifies the structure of GNN-based MARL approaches.

A generalized framework helps distinguish different GNN communication methods.

Abstract

In multi-agent reinforcement learning (MARL), the integration of a communication mechanism, allowing agents to better learn to coordinate their actions and converge on their objectives by sharing information. Based on an interaction graph, a subclass of methods employs graph neural networks (GNNs) to learn the communication, enabling agents to improve their internal representations by enriching them with information exchanged. With growing research, we note a lack of explicit structure and framework to distinguish and classify MARL approaches with communication based on GNNs. Thus, this paper surveys recent works in this field. We propose a generalized GNN-based communication process with the goal of making the underlying concepts behind the methods more obvious and accessible.