HONEST-CAV: Hierarchical Optimization of Network Signals and Trajectories for Connected and Automated Vehicles with Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a hierarchical multi-agent reinforcement learning framework for traffic control and vehicle trajectory optimization in mixed traffic, improving efficiency and energy use through decentralized control and ML-based trajectory planning.

Contribution

It proposes a novel hierarchical framework combining MARL and ML-based trajectory planning for CAVs and traffic signals, enhancing network efficiency and energy savings in mixed traffic environments.

Findings

MARL-based traffic signal control outperforms traditional methods in speed and fuel efficiency

MLTPA-guided CAVs further reduce energy consumption and idling time

60% CAV deployment improves average speed by 7.67%, fuel use by 10.23%, and idling time by 45.83%

Abstract

This study presents a hierarchical, network-level traffic flow control framework for mixed traffic consisting of Human-driven Vehicles (HVs), Connected and Automated Vehicles (CAVs). The framework jointly optimizes vehicle-level eco-driving behaviors and intersection-level traffic signal control to enhance overall network efficiency and decrease energy consumption. A decentralized Multi-Agent Reinforcement Learning (MARL) approach by Value Decomposition Network (VDN) manages cycle-based traffic signal control (TSC) at intersections, while an innovative Signal Phase and Timing (SPaT) prediction method integrates a Machine Learning-based Trajectory Planning Algorithm (MLTPA) to guide CAVs in executing Eco-Approach and Departure (EAD) maneuvers. The framework is evaluated across varying CAV proportions and powertrain types to assess its effects on mobility and energy performance. Experimental results conducted in a 4*4 real-world network demonstrate that the MARL-based TSC method outperforms the baseline model (i.e., Webster method) in speed, fuel consumption, and idling time. In addition, with MLTPA, HONEST-CAV benefits the traffic system further in energy consumption and idling time. With a 60% CAV proportion, vehicle average speed, fuel consumption, and idling time can be improved/saved by 7.67%, 10.23%, and 45.83% compared with the baseline. Furthermore, discussions on CAV proportions and powertrain types are conducted to quantify the performance of the proposed method with the impact of automation and electrification.