Artificial Generals Intelligence: Mastering Generals.io with Reinforcement Learning

TL;DR

This paper presents a new RTS environment based on Generals.io, a high-performance platform compatible with popular frameworks, and introduces a reference agent that achieves top-tier human performance through reinforcement learning techniques.

Contribution

The paper introduces a modular RTS benchmark environment and a competitive baseline agent, facilitating research in multi-agent reinforcement learning.

Findings

Agent reached top 0.003% of human leaderboard

Environment runs thousands of frames per second

Utilizes potential-based reward shaping and memory features

Abstract

We introduce a real-time strategy game environment based on Generals.io, a game with thousands of weekly active players. Our environment is fully compatible with Gymnasium and PettingZoo and is capable of running thousands of frames per second on commodity hardware. We also present a reference agent, trained with supervised pre-training and self-play, which reached the top 0.003% of the 1v1 human leaderboard after only 36 hours on a single H100 GPU. To accelerate learning, we incorporate potential-based reward shaping and memory features. Our contributions of a modular RTS benchmark and a competitive baseline agent provide an accessible yet challenging platform for advancing multi-agent reinforcement learning research. The documented code, together with examples and tutorials, is available at https://github.com/strakam/generals-bots.