Safety-Critical Scenario Generation Via Reinforcement Learning Based Editing

TL;DR

This paper introduces a reinforcement learning method for generating safety-critical scenarios in autonomous vehicle testing by sequential editing, effectively overcoming dimensionality issues and producing higher quality scenarios.

Contribution

It presents a novel deep reinforcement learning framework that uses sequential editing and generative models to generate diverse, plausible, and safety-critical scenarios.

Findings

Outperforms previous methods in scenario quality

Effectively explores high-dimensional scenario spaces

Incorporates risk and plausibility in reward design

Abstract

Generating safety-critical scenarios is essential for testing and verifying the safety of autonomous vehicles. Traditional optimization techniques suffer from the curse of dimensionality and limit the search space to fixed parameter spaces. To address these challenges, we propose a deep reinforcement learning approach that generates scenarios by sequential editing, such as adding new agents or modifying the trajectories of the existing agents. Our framework employs a reward function consisting of both risk and plausibility objectives. The plausibility objective leverages generative models, such as a variational autoencoder, to learn the likelihood of the generated parameters from the training datasets; It penalizes the generation of unlikely scenarios. Our approach overcomes the dimensionality challenge and explores a wide range of safety-critical scenarios. Our evaluation demonstrates that the proposed method generates safety-critical scenarios of higher quality compared with previous approaches.