High Performance Simulation for Scalable Multi-Agent Reinforcement Learning

TL;DR

This paper introduces Vogue, a high-performance multi-agent reinforcement learning framework that enables scalable training of thousands of agents on GPUs, facilitating the development of robust policies for complex system simulations.

Contribution

The paper presents Vogue, a GPU-based multi-agent environment supporting large-scale training of thousands of agents with high throughput, a significant advancement over existing limited-scale environments.

Findings

Vogue supports training thousands to tens of thousands of agents.

Training shared policies can be completed within minutes to hours.

Demonstrated high throughput and scalability in new multi-agent environments.

Abstract

Multi-agent reinforcement learning experiments and open-source training environments are typically limited in scale, supporting tens or sometimes up to hundreds of interacting agents. In this paper we demonstrate the use of Vogue, a high performance agent based model (ABM) framework. Vogue serves as a multi-agent training environment, supporting thousands to tens of thousands of interacting agents while maintaining high training throughput by running both the environment and reinforcement learning (RL) agents on the GPU. High performance multi-agent environments at this scale have the potential to enable the learning of robust and flexible policies for use in ABMs and simulations of complex systems. We demonstrate training performance with two newly developed, large scale multi-agent training environments. Moreover, we show that these environments can train shared RL policies on time-scales of minutes and hours.