Multi-Agent Deep Reinforcement Learning using Attentive Graph Neural Architectures for Real-Time Strategy Games

TL;DR

This paper introduces a novel multi-agent deep reinforcement learning algorithm for RTS games, combining QMIX, state categorization, and graph attention mechanisms to improve computational efficiency and agent coordination.

Contribution

It proposes the CSGA-policy, integrating state categorization and graph attention in MADRL for RTS games, enhancing performance and scalability.

Findings

CSGA-policy performs well in StarCraft II simulations.

The approach reduces computational complexity.

Effective agent relationship modeling via self-attention.

Abstract

In real-time strategy (RTS) game artificial intelligence research, various multi-agent deep reinforcement learning (MADRL) algorithms are widely and actively used nowadays. Most of the research is based on StarCraft II environment because it is the most well-known RTS games in world-wide. In our proposed MADRL-based algorithm, distributed MADRL is fundamentally used that is called QMIX. In addition to QMIX-based distributed computation, we consider state categorization which can reduce computational complexity significantly. Furthermore, self-attention mechanisms are used for identifying the relationship among agents in the form of graphs. Based on these approaches, we propose a categorized state graph attention policy (CSGA-policy). As observed in the performance evaluation of our proposed CSGA-policy with the most well-known StarCraft II simulation environment, our proposed algorithm works well in various settings, as expected.