Applying supervised and reinforcement learning methods to create neural-network-based agents for playing StarCraft II

TL;DR

This paper introduces a neural network-based agent for StarCraft II trained with supervised and reinforcement learning on a single GPU, achieving moderate performance without extensive expert knowledge or large computational resources.

Contribution

The authors present a novel neural network architecture for StarCraft II that can be trained with general-purpose methods on consumer hardware, serving as a baseline for small-scale RTS research.

Findings

Achieves non-trivial performance against scripted bots

Requires only a single GPU and minimal expert knowledge

Can be adapted to other RTS games with minor modifications

Abstract

Recently, multiple approaches for creating agents for playing various complex real-time computer games such as StarCraft II or Dota 2 were proposed, however, they either embed a significant amount of expert knowledge into the agent or use a prohibitively large for most researchers amount of computational resources. We propose a neural network architecture for playing the full two-player match of StarCraft II trained with general-purpose supervised and reinforcement learning, that can be trained on a single consumer-grade PC with a single GPU. We also show that our implementation achieves a non-trivial performance when compared to the in-game scripted bots. We make no simplifying assumptions about the game except for playing on a single chosen map, and we use very little expert knowledge. In principle, our approach can be applied to any RTS game with small modifications. While our results are far behind the state-of-the-art large-scale approaches in terms of the final performance, we believe our work can serve as a solid baseline for other small-scale experiments.