Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control

TL;DR

This paper introduces a hybrid car-following strategy combining deep reinforcement learning (DDPG) and cooperative adaptive cruise control (CACC) to enhance performance in complex driving environments, addressing limitations of existing methods.

Contribution

It proposes a novel hybrid approach that integrates DDPG and CACC, selecting the best action based on reward to improve car-following performance.

Findings

Improved car-following accuracy compared to standalone DDPG and CACC.

Enhanced stability and responsiveness in simulated driving scenarios.

Effective balance between exploration and rule-based control.

Abstract

Deep deterministic policy gradient (DDPG)-based car-following strategy can break through the constraints of the differential equation model due to the ability of exploration on complex environments. However, the car-following performance of DDPG is usually degraded by unreasonable reward function design, insufficient training, and low sampling efficiency. In order to solve this kind of problem, a hybrid car-following strategy based on DDPG and cooperative adaptive cruise control (CACC) is proposed. First, the car-following process is modeled as the Markov decision process to calculate CACC and DDPG simultaneously at each frame. Given a current state, two actions are obtained from CACC and DDPG, respectively. Then, an optimal action, corresponding to the one offering a larger reward, is chosen as the output of the hybrid strategy. Meanwhile, a rule is designed to ensure that the change rate of acceleration is smaller than the desired value. Therefore, the proposed strategy not only guarantees the basic performance of car-following through CACC but also makes full use of the advantages of exploration on complex environments via DDPG. Finally, simulation results show that the car-following performance of the proposed strategy is improved compared with that of DDPG and CACC.