Massively Multiagent Minigames for Training Generalist Agents

TL;DR

Meta MMO introduces a collection of multiagent minigames designed as a reinforcement learning benchmark to evaluate and improve generalization of agents across diverse multiagent tasks.

Contribution

The paper expands Neural MMO with multiple efficient minigames and demonstrates learning generalization across them with a single model.

Findings

Successfully trained agents that generalize across multiple minigames

Provides a new benchmark for multiagent reinforcement learning

Open-sourced environment and code for community use

Abstract

We present Meta MMO, a collection of many-agent minigames for use as a reinforcement learning benchmark. Meta MMO is built on top of Neural MMO, a massively multiagent environment that has been the subject of two previous NeurIPS competitions. Our work expands Neural MMO with several computationally efficient minigames. We explore generalization across Meta MMO by learning to play several minigames with a single set of weights. We release the environment, baselines, and training code under the MIT license. We hope that Meta MMO will spur additional progress on Neural MMO and, more generally, will serve as a useful benchmark for many-agent generalization.