Controllable risk scenario generation from human crash data for autonomous vehicle testing

TL;DR

This paper presents CRAG, a framework for generating realistic and controllable risk scenarios in autonomous vehicle testing by modeling and transitioning between nominal and risk-prone behaviors using a structured latent space.

Contribution

CRAG introduces a novel structured latent space and mode-transition mechanism to generate diverse, controllable risk scenarios from limited crash data for AV safety testing.

Findings

CRAG improves diversity of generated scenarios.

CRAG enables controllable risk scenario generation.

CRAG enhances evaluation of AV robustness.

Abstract

Ensuring the safety of autonomous vehicles (AV) requires rigorous testing under both everyday driving and rare, safety-critical conditions. A key challenge lies in simulating environment agents, including background vehicles (BVs) and vulnerable road users (VRUs), that behave realistically in nominal traffic while also exhibiting risk-prone behaviors consistent with real-world accidents. We introduce Controllable Risk Agent Generation (CRAG), a framework designed to unify the modeling of dominant nominal behaviors and rare safety-critical behaviors. CRAG constructs a structured latent space that disentangles normal and risk-related behaviors, enabling efficient use of limited crash data. By combining risk-aware latent representations with optimization-based mode-transition mechanisms, the framework allows agents to shift smoothly and plausibly from safe to risk states over extended horizons, while maintaining high fidelity in both regimes. Extensive experiments show that CRAG improves diversity compared to existing baselines, while also enabling controllable generation of risk scenarios for targeted and efficient evaluation of AV robustness.