Deep Multi-Agent Reinforcement Learning with Relevance Graphs

TL;DR

This paper introduces MAGnet, a novel multi-agent reinforcement learning method that uses relevance graphs and self-attention to improve performance, demonstrating significant gains in the Pommerman game over existing methods.

Contribution

The paper presents MAGnet, a new MARL approach combining relevance graphs and self-attention, achieving superior results in complex multi-agent environments.

Findings

MAGnet outperforms DQN, MADDPG, and MCTS in Pommerman.

Relevance graphs improve environment understanding in MARL.

Self-attention enhances communication among agents.

Abstract

Over recent years, deep reinforcement learning has shown strong successes in complex single-agent tasks, and more recently this approach has also been applied to multi-agent domains. In this paper, we propose a novel approach, called MAGnet, to multi-agent reinforcement learning (MARL) that utilizes a relevance graph representation of the environment obtained by a self-attention mechanism, and a message-generation technique inspired by the NerveNet architecture. We applied our MAGnet approach to the Pommerman game and the results show that it significantly outperforms state-of-the-art MARL solutions, including DQN, MADDPG, and MCTS.