BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Prediction

TL;DR

BehaviorGPT is a fully autoregressive Transformer-based simulator that models traffic agent behaviors without separating history and future, achieving high realism and efficiency in autonomous driving simulations.

Contribution

It introduces a homogeneous, autoregressive Transformer and the Next-Patch Prediction paradigm to improve traffic agent simulation accuracy and data efficiency.

Findings

Won first place in Waymo Open Sim Agents Challenge 2024

Achieved a realism score of 0.7473

Attained a minADE score of 1.4147

Abstract

Simulating realistic behaviors of traffic agents is pivotal for efficiently validating the safety of autonomous driving systems. Existing data-driven simulators primarily use an encoder-decoder architecture to encode the historical trajectories before decoding the future. However, the heterogeneity between encoders and decoders complicates the models, and the manual separation of historical and future trajectories leads to low data utilization. Given these limitations, we propose BehaviorGPT, a homogeneous and fully autoregressive Transformer designed to simulate the sequential behavior of multiple agents. Crucially, our approach discards the traditional separation between "history" and "future" by modeling each time step as the "current" one for motion generation, leading to a simpler, more parameter- and data-efficient agent simulator. We further introduce the Next-Patch Prediction Paradigm (NP3) to mitigate the negative effects of autoregressive modeling, in which models are trained to reason at the patch level of trajectories and capture long-range spatial-temporal interactions. Despite having merely 3M model parameters, BehaviorGPT won first place in the 2024 Waymo Open Sim Agents Challenge with a realism score of 0.7473 and a minADE score of 1.4147, demonstrating its exceptional performance in traffic agent simulation.