Continual Driving Policy Optimization with Closed-Loop Individualized Curricula

TL;DR

This paper introduces CLIC, a continual driving policy optimization framework that iteratively improves autonomous vehicle safety by reusing and tailoring scenario libraries through a closed-loop, individualized curriculum approach.

Contribution

The paper proposes a novel framework that reuses extensive scenario libraries for AV training by estimating failure probabilities and customizing curricula, enhancing safety-critical scenario handling.

Findings

CLIC outperforms existing curriculum strategies in risky scenario management.

It effectively reuses large scenario libraries for continuous AV improvement.

CLIC maintains proficiency in simpler driving scenarios.

Abstract

The safety of autonomous vehicles (AV) has been a long-standing top concern, stemming from the absence of rare and safety-critical scenarios in the long-tail naturalistic driving distribution. To tackle this challenge, a surge of research in scenario-based autonomous driving has emerged, with a focus on generating high-risk driving scenarios and applying them to conduct safety-critical testing of AV models. However, limited work has been explored on the reuse of these extensive scenarios to iteratively improve AV models. Moreover, it remains intractable and challenging to filter through gigantic scenario libraries collected from other AV models with distinct behaviors, attempting to extract transferable information for current AV improvement. Therefore, we develop a continual driving policy optimization framework featuring Closed-Loop Individualized Curricula (CLIC), which we factorize into a set of standardized sub-modules for flexible implementation choices: AV Evaluation, Scenario Selection, and AV Training. CLIC frames AV Evaluation as a collision prediction task, where it estimates the chance of AV failures in these scenarios at each iteration. Subsequently, by re-sampling from historical scenarios based on these failure probabilities, CLIC tailors individualized curricula for downstream training, aligning them with the evaluated capability of AV. Accordingly, CLIC not only maximizes the utilization of the vast pre-collected scenario library for closed-loop driving policy optimization but also facilitates AV improvement by individualizing its training with more challenging cases out of those poorly organized scenarios. Experimental results clearly indicate that CLIC surpasses other curriculum-based training strategies, showing substantial improvement in managing risky scenarios, while still maintaining proficiency in handling simpler cases.