Communication-Aware Reinforcement Learning for Cooperative Adaptive Cruise Control

TL;DR

This paper introduces Communication-Aware Reinforcement Learning (CA-RL), a novel approach for cooperative adaptive cruise control that enhances scalability and robustness in multi-vehicle traffic scenarios by incorporating communication modules.

Contribution

The paper proposes CA-RL, a communication-aware MARL framework that improves scalability and performance in CACC by efficient information exchange among vehicles.

Findings

CA-RL outperforms baseline methods in various traffic scenarios.

It maintains high performance despite changes in vehicle numbers.

CA-RL enhances system robustness and scalability.

Abstract

Cooperative Adaptive Cruise Control (CACC) plays a pivotal role in enhancing traffic efficiency and safety in Connected and Autonomous Vehicles (CAVs). Reinforcement Learning (RL) has proven effective in optimizing complex decision-making processes in CACC, leading to improved system performance and adaptability. Among RL approaches, Multi-Agent Reinforcement Learning (MARL) has shown remarkable potential by enabling coordinated actions among multiple CAVs through Centralized Training with Decentralized Execution (CTDE). However, MARL often faces scalability issues, particularly when CACC vehicles suddenly join or leave the platoon, resulting in performance degradation. To address these challenges, we propose Communication-Aware Reinforcement Learning (CA-RL). CA-RL includes a communication-aware module that extracts and compresses vehicle communication information through forward and backward information transmission modules. This enables efficient cyclic information propagation within the CACC traffic flow, ensuring policy consistency and mitigating the scalability problems of MARL in CACC. Experimental results demonstrate that CA-RL significantly outperforms baseline methods in various traffic scenarios, achieving superior scalability, robustness, and overall system performance while maintaining reliable performance despite changes in the number of participating vehicles.