ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

TL;DR

ELF is a flexible, lightweight platform for reinforcement learning research in real-time strategy games, enabling fast training and testing of AI agents with customizable environments and game parameters.

Contribution

We introduce ELF, a highly customizable RTS research platform that supports fast training, flexible environment-agent configurations, and integration with existing game environments.

Findings

Mini-RTS runs at 40K FPS per core on a Macbook Pro.

Full-game bots trained in one day using modern RL methods.

Our RL agents outperform rule-based AI in Mini-RTS over 70% of the time.

Abstract

In this paper, we propose ELF, an Extensive, Lightweight and Flexible platform for fundamental reinforcement learning research. Using ELF, we implement a highly customizable real-time strategy (RTS) engine with three game environments (Mini-RTS, Capture the Flag and Tower Defense). Mini-RTS, as a miniature version of StarCraft, captures key game dynamics and runs at 40K frame-per-second (FPS) per core on a Macbook Pro notebook. When coupled with modern reinforcement learning methods, the system can train a full-game bot against built-in AIs end-to-end in one day with 6 CPUs and 1 GPU. In addition, our platform is flexible in terms of environment-agent communication topologies, choices of RL methods, changes in game parameters, and can host existing C/C++-based game environments like Arcade Learning Environment. Using ELF, we thoroughly explore training parameters and show that a network with Leaky ReLU and Batch Normalization coupled with long-horizon training and progressive curriculum beats the rule-based built-in AI more than $70\%$ of the time in the full game of Mini-RTS. Strong performance is also achieved on the other two games. In game replays, we show our agents learn interesting strategies. ELF, along with its RL platform, is open-sourced at https://github.com/facebookresearch/ELF.