Towards Heterogeneous Multi-Agent Reinforcement Learning with Graph Neural Networks

TL;DR

This paper introduces a neural network architecture leveraging graph neural networks to learn policies for heterogeneous multi-agent systems, effectively modeling diverse communication channels and entity classes to improve performance.

Contribution

It presents a novel GNN-based architecture that encodes heterogeneous entities and their interactions, enabling specialized communication channels for multi-agent reinforcement learning.

Findings

Specialized communication channels improve performance.

Graph neural networks effectively model heterogeneous agent interactions.

Parameter sharing enhances learning efficiency.

Abstract

This work proposes a neural network architecture that learns policies for multiple agent classes in a heterogeneous multi-agent reinforcement setting. The proposed network uses directed labeled graph representations for states, encodes feature vectors of different sizes for different entity classes, uses relational graph convolution layers to model different communication channels between entity types and learns distinct policies for different agent classes, sharing parameters wherever possible. Results have shown that specializing the communication channels between entity classes is a promising step to achieve higher performance in environments composed of heterogeneous entities.