Marathon Environments: Multi-Agent Continuous Control Benchmarks in a Modern Video Game Engine

TL;DR

Marathon Environments introduces open-source, Unity-based continuous control benchmarks for multi-agent reinforcement learning, demonstrating transferability of research to commercial game engines and showcasing diverse complex control tasks.

Contribution

This work provides the first open-source, Unity-based benchmark suite for continuous control, enabling transfer of research to commercial game engines and demonstrating robustness across multiple tasks.

Findings

Successfully reproduced advanced control tasks like walking, running, and backflipping.

Validated transferability of continuous control algorithms to Unity engine.

Reduced training time with effective strategies.

Abstract

Recent advances in deep reinforcement learning in the paradigm of locomotion using continuous control have raised the interest of game makers for the potential of digital actors using active ragdoll. Currently, the available options to develop these ideas are either researchers' limited codebase or proprietary closed systems. We present Marathon Environments, a suite of open source, continuous control benchmarks implemented on the Unity game engine, using the Unity ML- Agents Toolkit. We demonstrate through these benchmarks that continuous control research is transferable to a commercial game engine. Furthermore, we exhibit the robustness of these environments by reproducing advanced continuous control research, such as learning to walk, run and backflip from motion capture data; learning to navigate complex terrains; and by implementing a video game input control system. We show further robustness by training with alternative algorithms found in OpenAI.Baselines. Finally, we share strategies for significantly reducing the training time.