Heterogeneous Self-Play for Realistic Highway Traffic Simulation

TL;DR

PHASE is a self-play framework for realistic highway traffic simulation that enables controllable, broad, and behaviorally credible scenario generation, improving safety evaluation for autonomous vehicles.

Contribution

The paper introduces PHASE, a novel self-play method that generates diverse, controllable, and realistic highway scenarios including multiple vehicle types, with zero-shot transfer to real-world data.

Findings

PHASE achieves a 96.3% success rate on 512 unseen real scenarios.

Reduces trajectory prediction errors from 6.57/12.07 m to 2.44/5.25 m.

Improves behavioral realism metrics by over 13%.

Abstract

Realistic highway simulation is critical for scalable safety evaluation of autonomous vehicles, particularly for interactions that are too rare to study from logged data alone. Yet highway traffic generation remains challenging because it requires broad coverage across speeds and maneuvers, controllable generation of rare safety-critical scenarios, and behavioral credibility in multi-agent interactions. We present PHASE, Policy for Heterogeneous Agent Self-play on Expressway, a context-aware self-play framework that addresses these three requirements through explicit per-agent conditioning for controllability, synthetic scenario generation for broad highway coverage, and closed-loop multi-agent training for realistic interaction dynamics. PHASE further supports different vehicle profiles, for example, passenger cars and articulated trailer trucks, within a single policy via vehicle-aware dynamics and context-conditioned actions, and stabilizes self-play with early termination of unrecoverable states, at-fault collision attribution, highway-aware reward shaping, coupled curricula, and robust policy optimization. Despite being trained only on synthetic data, PHASE transfers zero-shot to 512 unseen high-interaction real scenarios in exiD, achieving a 96.3% success rate and reducing ADE/FDE from 6.57/12.07 m to 2.44/5.25 m relative to a prior self-play baseline. In a learned trajectory embedding space, it also improves behavioral realism over IDM, reducing Frechet trajectory distance by 13.1% and energy distance by 20.2%. These results show that synthetic self-play can provide a scalable route to controllable and realistic highway scenario generation without direct imitation of expert logs.