A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning

TL;DR

This paper introduces a hierarchical planning framework combining safe low-level controllers with a high-level reinforcement learning coordinator to enable autonomous vehicles to make safe, efficient decisions in complex traffic scenarios.

Contribution

It presents a novel hierarchical behavior planning framework with a reinforcement learning high-level controller and safety-guaranteed low-level controllers for autonomous driving.

Findings

Effective in diverse traffic scenarios

Ensures safety while maintaining efficiency

Outperforms traditional planning methods

Abstract

Autonomous vehicles need to handle various traffic conditions and make safe and efficient decisions and maneuvers. However, on the one hand, a single optimization/sampling-based motion planner cannot efficiently generate safe trajectories in real time, particularly when there are many interactive vehicles near by. On the other hand, end-to-end learning methods cannot assure the safety of the outcomes. To address this challenge, we propose a hierarchical behavior planning framework with a set of low-level safe controllers and a high-level reinforcement learning algorithm (H-CtRL) as a coordinator for the low-level controllers. Safety is guaranteed by the low-level optimization/sampling-based controllers, while the high-level reinforcement learning algorithm makes H-CtRL an adaptive and efficient behavior planner. To train and test our proposed algorithm, we built a simulator that can reproduce traffic scenes using real-world datasets. The proposed H-CtRL is proved to be effective in various realistic simulation scenarios, with satisfying performance in terms of both safety and efficiency.