Multi-agent DRL-based Lane Change Decision Model for Cooperative Planning in Mixed Traffic

TL;DR

This paper introduces a multi-agent deep reinforcement learning model using QMIX and CNNs to improve cooperative lane-changing in mixed traffic, significantly increasing platooning rates during early CAV deployment.

Contribution

It presents a novel hybrid multi-agent decision model that adapts to varying numbers of CAVs in mixed traffic using CNN-QMIX architecture, enhancing cooperative planning.

Findings

Increases cooperative platooning rates up to 26.2%.

Outperforms baseline rule-based models in mixed traffic scenarios.

Effectively manages fluctuating numbers of CAVs in simulation.

Abstract

Connected automated vehicles (CAVs) possess the ability to communicate and coordinate with one another, enabling cooperative platooning that enhances both energy efficiency and traffic flow. However, during the initial stage of CAV deployment, the sparse distribution of CAVs among human-driven vehicles reduces the likelihood of forming effective cooperative platoons. To address this challenge, this study proposes a hybrid multi-agent lane change decision model aimed at increasing CAV participation in cooperative platooning and maximizing its associated benefits. The proposed model employs the QMIX framework, integrating traffic data processed through a convolutional neural network (CNN-QMIX). This architecture addresses a critical issue in dynamic traffic scenarios by enabling CAVs to make optimal decisions irrespective of the varying number of CAVs present in mixed traffic. Additionally, a trajectory planner and a model predictive controller are designed to ensure smooth and safe lane-change execution. The proposed model is trained and evaluated within a microsimulation environment under varying CAV market penetration rates. The results demonstrate that the proposed model efficiently manages fluctuating traffic agent numbers, significantly outperforming the baseline rule-based models. Notably, it enhances cooperative platooning rates up to 26.2\%, showcasing its potential to optimize CAV cooperation and traffic dynamics during the early stage of deployment.