ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stable

TL;DR

ReGentS is a novel framework for generating realistic safety-critical driving scenarios from real-world data, using trajectory optimization and a differentiable simulator to improve autonomous driving training.

Contribution

It introduces a stable trajectory generation method that handles complex multi-agent scenarios and avoids unrealistic or collision scenarios, enhancing safety-critical data synthesis.

Findings

Successfully generates diverse safety-critical scenarios with up to 32 agents.

Stabilizes trajectories to prevent unrealistic diverging paths.

Utilizes a differentiable simulator for efficient gradient-based optimization.

Abstract

Machine learning based autonomous driving systems often face challenges with safety-critical scenarios that are rare in real-world data, hindering their large-scale deployment. While increasing real-world training data coverage could address this issue, it is costly and dangerous. This work explores generating safety-critical driving scenarios by modifying complex real-world regular scenarios through trajectory optimization. We propose ReGentS, which stabilizes generated trajectories and introduces heuristics to avoid obvious collisions and optimization problems. Our approach addresses unrealistic diverging trajectories and unavoidable collision scenarios that are not useful for training robust planner. We also extend the scenario generation framework to handle real-world data with up to 32 agents. Additionally, by using a differentiable simulator, our approach simplifies gradient descent-based optimization involving a simulator, paving the way for future advancements. The code is available at https://github.com/valeoai/ReGentS.