Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks

TL;DR

Neural MMO v1.3 introduces a complex multiagent game environment inspired by MMORPGs, enabling research on multiagent learning, with demonstrated emergent behaviors using standard reinforcement learning methods.

Contribution

The paper presents Neural MMO v1.3, a scalable multiagent environment modeling MMORPG complexities, and discusses infrastructure and IO challenges for AI research.

Findings

Standard policy gradient methods learn interesting emergent behaviors.

Neural MMO captures complexities of real-world learning environments.

Progress on distributed infrastructure and game IO challenges.

Abstract

Progress in multiagent intelligence research is fundamentally limited by the number and quality of environments available for study. In recent years, simulated games have become a dominant research platform within reinforcement learning, in part due to their accessibility and interpretability. Previous works have targeted and demonstrated success on arcade, first person shooter (FPS), real-time strategy (RTS), and massive online battle arena (MOBA) games. Our work considers massively multiplayer online role-playing games (MMORPGs or MMOs), which capture several complexities of real-world learning that are not well modeled by any other game genre. We present Neural MMO, a massively multiagent game environment inspired by MMOs and discuss our progress on two more general challenges in multiagent systems engineering for AI research: distributed infrastructure and game IO. We further demonstrate that standard policy gradient methods and simple baseline models can learn interesting emergent exploration and specialization behaviors in this setting.