Dynamically Expanding Capacity of Autonomous Driving with Near-Miss Focused Training Framework

TL;DR

This paper introduces a near-miss focused training framework for autonomous vehicles that uses simulated scenarios and adversarial reinforcement learning to improve safety and robustness in rare critical situations.

Contribution

It proposes a novel reward function and a RARL-based training framework to generate near-miss scenarios and enhance AV safety capabilities.

Findings

Generated scenarios closer to near-miss events

Enhanced AV and BV capabilities through training

Improved safety performance in simulated tests

Abstract

The long-tail distribution of real driving data poses challenges for training and testing autonomous vehicles (AV), where rare yet crucial safety-critical scenarios are infrequent. And virtual simulation offers a low-cost and efficient solution. This paper proposes a near-miss focused training framework for AV. Utilizing the driving scenario information provided by sensors in the simulator, we design novel reward functions, which enable background vehicles (BV) to generate near-miss scenarios and ensure gradients exist not only in collision-free scenes but also in collision scenarios. And then leveraging the Robust Adversarial Reinforcement Learning (RARL) framework for simultaneous training of AV and BV to gradually enhance AV and BV capabilities, as well as generating near-miss scenarios tailored to different levels of AV capabilities. Results from three testing strategies indicate that the proposed method generates scenarios closer to near-miss, thus enhancing the capabilities of both AVs and BVs throughout training.