Specializing Inter-Agent Communication in Heterogeneous Multi-Agent Reinforcement Learning using Agent Class Information

TL;DR

This paper introduces a novel graph-based framework for specialized inter-agent communication in heterogeneous multi-agent reinforcement learning, enabling improved cooperation by learning class-specific message transformations.

Contribution

It proposes a directed labeled heterogeneous agent graph and a neural architecture that learns class-specific message transformations for better communication.

Findings

Achieves comparable or superior performance in multi-agent environments with many agent classes.

Demonstrates effectiveness of class-specific communication transformations.

Employs parameter sharing for heterogeneous agents to enhance scalability.

Abstract

Inspired by recent advances in agent communication with graph neural networks, this work proposes the representation of multi-agent communication capabilities as a directed labeled heterogeneous agent graph, in which node labels denote agent classes and edge labels, the communication type between two classes of agents. We also introduce a neural network architecture that specializes communication in fully cooperative heterogeneous multi-agent tasks by learning individual transformations to the exchanged messages between each pair of agent classes. By also employing encoding and action selection modules with parameter sharing for environments with heterogeneous agents, we demonstrate comparable or superior performance in environments where a larger number of agent classes operates.