Scenario-Based Hierarchical Reinforcement Learning for Automated Driving Decision Making

TL;DR

This paper introduces SAD-RL, a hierarchical reinforcement learning framework for automated driving decision-making that improves safety and efficiency in complex scenarios by integrating scenario-based training with hierarchical policies.

Contribution

The paper presents the first scenario-based hierarchical RL framework for automated driving, enhancing generalizability and learning efficiency in complex environments.

Findings

Agents trained with SAD-RL achieve safe behavior in diverse situations.

Hierarchical policies and scenario diversity are crucial for performance.

SAD-RL improves learning efficiency in complex driving tasks.

Abstract

Developing decision-making algorithms for highly automated driving systems remains challenging, since these systems have to operate safely in an open and complex environments. Reinforcement Learning (RL) approaches can learn comprehensive decision policies directly from experience and already show promising results in simple driving tasks. However, current approaches fail to achieve generalizability for more complex driving tasks and lack learning efficiency. Therefore, we present Scenario-based Automated Driving Reinforcement Learning (SAD-RL), the first framework that integrates Reinforcement Learning (RL) of hierarchical policy in a scenario-based environment. A high-level policy selects maneuver templates that are evaluated and executed by a low-level control logic. The scenario-based environment allows to control the training experience for the agent and to explicitly introduce challenging, but rate situations into the training process. Our experiments show that an agent trained using the SAD-RL framework can achieve safe behaviour in easy as well as challenging situations efficiently. Our ablation studies confirmed that both HRL and scenario diversity are essential for achieving these results.