Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learning

TL;DR

This paper develops a deep reinforcement learning-based decision-making policy for autonomous highway driving, focusing on overtaking maneuvers to enhance safety and efficiency.

Contribution

It introduces a hierarchical control framework combined with a dueling deep Q-network algorithm for improved highway decision-making in autonomous vehicles.

Findings

The DDQN-based policy effectively performs overtaking maneuvers.

The proposed method improves convergence rate and control performance.

Simulation results demonstrate safe and efficient highway driving.

Abstract

Autonomous driving is a promising technology to reduce traffic accidents and improve driving efficiency. In this work, a deep reinforcement learning (DRL)-enabled decision-making policy is constructed for autonomous vehicles to address the overtaking behaviors on the highway. First, a highway driving environment is founded, wherein the ego vehicle aims to pass through the surrounding vehicles with an efficient and safe maneuver. A hierarchical control framework is presented to control these vehicles, which indicates the upper-level manages the driving decisions, and the lower-level cares about the supervision of vehicle speed and acceleration. Then, the particular DRL method named dueling deep Q-network (DDQN) algorithm is applied to derive the highway decision-making strategy. The exhaustive calculative procedures of deep Q-network and DDQN algorithms are discussed and compared. Finally, a series of estimation simulation experiments are conducted to evaluate the effectiveness of the proposed highway decision-making policy. The advantages of the proposed framework in convergence rate and control performance are illuminated. Simulation results reveal that the DDQN-based overtaking policy could accomplish highway driving tasks efficiently and safely.