Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach

TL;DR

This paper introduces a hierarchical reinforcement learning approach using motion primitives to develop efficient and high-performing autonomous driving strategies in dense traffic scenarios.

Contribution

It proposes a skill-based hierarchical framework that reduces training time and improves performance of autonomous driving models compared to traditional reinforcement learning.

Findings

Higher performance with less training time

Effective in dense traffic merging scenarios

Utilizes motion primitives as high-level actions

Abstract

Driving in dense traffic with human and autonomous drivers is a challenging task that requires high-level planning and reasoning. Human drivers can achieve this task comfortably, and there has been many efforts to model human driver strategies. These strategies can be used as inspirations for developing autonomous driving algorithms or to create high-fidelity simulators. Reinforcement learning is a common tool to model driver policies, but conventional training of these models can be computationally expensive and time-consuming. To address this issue, in this paper, we propose ``skill-based" hierarchical driving strategies, where motion primitives, i.e. skills, are designed and used as high-level actions. This reduces the training time for applications that require multiple models with varying behavior. Simulation results in a merging scenario demonstrate that the proposed approach yields driver models that achieve higher performance with less training compared to baseline reinforcement learning methods.