Versatile Behavior Diffusion for Generalized Traffic Agent Simulation

TL;DR

The paper presents Versatile Behavior Diffusion (VBD), a diffusion-based traffic simulation framework that generates realistic, controllable multi-agent interactions, including safety-critical scenarios, to improve autonomous vehicle testing.

Contribution

VBD introduces a novel diffusion model for traffic scenario generation with scene consistency, multi-step scenario editing, and the ability to create safety-critical situations from normal data.

Findings

VBD achieves realistic multi-agent traffic simulations.

It enables flexible scenario editing and long-tail safety-critical scenario generation.

Experimental results demonstrate its effectiveness and versatility.

Abstract

Existing traffic simulation models often fall short in capturing the intricacies of real-world scenarios, particularly the interactive behaviors among multiple traffic participants, thereby limiting their utility in the evaluation and validation of autonomous driving systems. We introduce Versatile Behavior Diffusion (VBD), a novel traffic scenario generation framework based on diffusion generative models that synthesizes scene-consistent, realistic, and controllable multi-agent interactions. VBD achieves strong performance in closed-loop traffic simulation, generating scene-consistent agent behaviors that reflect complex agent interactions. A key capability of VBD is inference-time scenario editing through multi-step refinement, guided by behavior priors and model-based optimization objectives, enabling flexible and controllable behavior generation. Despite being trained on real-world traffic datasets with only normal conditions, we introduce conflict-prior and game-theoretic guidance approaches. These approaches enable the generation of interactive, customizable, or long-tail safety-critical scenarios, which are essential for comprehensive testing and validation of autonomous driving systems. Extensive experiments validate the effectiveness and versatility of VBD and highlight its promise as a foundational tool for advancing traffic simulation and autonomous vehicle development. Project website: https://sites.google.com/view/versatile-behavior-diffusion