GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS

TL;DR

GPUDrive is a GPU-accelerated multi-agent simulator capable of over a million simulation steps per second, enabling scalable reinforcement learning for complex multi-agent planning tasks.

Contribution

The paper introduces GPUDrive, a high-performance, flexible multi-agent simulation platform that significantly accelerates the training and evaluation of multi-agent systems.

Findings

Achieved over 1 million simulation steps per second.

Enabled training of RL agents on large-scale datasets in minutes.

Demonstrated scalable multi-agent planning with complex behaviors.

Abstract

Multi-agent learning algorithms have been successful at generating superhuman planning in various games but have had limited impact on the design of deployed multi-agent planners. A key bottleneck in applying these techniques to multi-agent planning is that they require billions of steps of experience. To enable the study of multi-agent planning at scale, we present GPUDrive. GPUDrive is a GPU-accelerated, multi-agent simulator built on top of the Madrona Game Engine capable of generating over a million simulation steps per second. Observation, reward, and dynamics functions are written directly in C++, allowing users to define complex, heterogeneous agent behaviors that are lowered to high-performance CUDA. Despite these low-level optimizations, GPUDrive is fully accessible through Python, offering a seamless and efficient workflow for multi-agent, closed-loop simulation. Using GPUDrive, we train reinforcement learning agents on the Waymo Open Motion Dataset, achieving efficient goal-reaching in minutes and scaling to thousands of scenarios in hours. We open-source the code and pre-trained agents at https://github.com/Emerge-Lab/gpudrive.