Partially Connected Automated Vehicle Cooperative Control Strategy with a Deep Reinforcement Learning Approach

TL;DR

This paper introduces a deep reinforcement learning-based cooperative control strategy for partially connected automated vehicles, improving traffic stability, efficiency, and energy savings in mixed traffic environments.

Contribution

It develops a novel DRL-based control method that decomposes mixed traffic into subsystems to enhance learning efficiency and system performance.

Findings

Enhanced string stability in mixed traffic flows.

Improved car following efficiency and energy savings.

Good generalization across different traffic scenarios.

Abstract

This paper proposes a cooperative strategy of connected and automated vehicles (CAVs) longitudinal control for partially connected and automated traffic environment based on deep reinforcement learning (DRL) algorithm, which enhances the string stability of mixed traffic, car following efficiency, and energy efficiency. Since the sequences of mixed traffic are combinatory, to reduce the training dimension and alleviate communication burdens, we decomposed mixed traffic into multiple subsystems where each subsystem is comprised of human-driven vehicles (HDV) followed by cooperative CAVs. Based on that, a cooperative CAV control strategy is developed based on a deep reinforcement learning algorithm, enabling CAVs to learn the leading HDV's characteristics and make longitudinal control decisions cooperatively to improve the performance of each subsystem locally and consequently enhance performance for the whole mixed traffic flow. For training, a distributed proximal policy optimization is applied to ensure the training convergence of the proposed DRL. To verify the effectiveness of the proposed method, simulated experiments are conducted, which shows the performance of our proposed model has a great generalization capability of dampening oscillations, fulfilling the car following and energy-saving tasks efficiently under different penetration rates and various leading HDVs behaviors. Keywords: partially connected automated traffic environment, cooperative control, deep reinforcement learning, traffic oscillation dampening, energy efficiency