Multi-Agent Game Abstraction via Graph Attention Neural Network

TL;DR

This paper introduces G2ANet, a graph attention neural network that models local interactions in large-scale multi-agent systems, simplifying policy learning and improving performance.

Contribution

It proposes a novel two-stage attention mechanism for game abstraction and integrates it into multi-agent reinforcement learning algorithms.

Findings

Improved asymptotic performance over state-of-the-art methods

Effective simplification of the learning process in large-scale systems

Successful application in Traffic Junction and Predator-Prey environments

Abstract

In large-scale multi-agent systems, the large number of agents and complex game relationship cause great difficulty for policy learning. Therefore, simplifying the learning process is an important research issue. In many multi-agent systems, the interactions between agents often happen locally, which means that agents neither need to coordinate with all other agents nor need to coordinate with others all the time. Traditional methods attempt to use pre-defined rules to capture the interaction relationship between agents. However, the methods cannot be directly used in a large-scale environment due to the difficulty of transforming the complex interactions between agents into rules. In this paper, we model the relationship between agents by a complete graph and propose a novel game abstraction mechanism based on two-stage attention network (G2ANet), which can indicate whether there is an interaction between two agents and the importance of the interaction. We integrate this detection mechanism into graph neural network-based multi-agent reinforcement learning for conducting game abstraction and propose two novel learning algorithms GA-Comm and GA-AC. We conduct experiments in Traffic Junction and Predator-Prey. The results indicate that the proposed methods can simplify the learning process and meanwhile get better asymptotic performance compared with state-of-the-art algorithms.