MazeBase: A Sandbox for Learning from Games

TL;DR

MazeBase is a versatile 2D game environment designed for training and evaluating machine learning models on reasoning and planning tasks, demonstrating transferability to complex real-time strategy scenarios.

Contribution

Introduces MazeBase, a new sandbox environment with diverse tasks for machine learning research, and shows its effectiveness in training models that transfer to StarCraft scenarios.

Findings

Models perform suboptimally on simple tasks, indicating room for improvement.

Reinforcement learning models trained in MazeBase can beat in-game AI in StarCraft scenarios.

MazeBase effectively emulates small-scale combat scenarios for transfer learning.

Abstract

This paper introduces MazeBase: an environment for simple 2D games, designed as a sandbox for machine learning approaches to reasoning and planning. Within it, we create 10 simple games embodying a range of algorithmic tasks (e.g. if-then statements or set negation). A variety of neural models (fully connected, convolutional network, memory network) are deployed via reinforcement learning on these games, with and without a procedurally generated curriculum. Despite the tasks' simplicity, the performance of the models is far from optimal, suggesting directions for future development. We also demonstrate the versatility of MazeBase by using it to emulate small combat scenarios from StarCraft. Models trained on the MazeBase version can be directly applied to StarCraft, where they consistently beat the in-game AI.