Fault-Tolerant MARL for CAVs under Observation Perturbations for Highway On-Ramp Merging

TL;DR

This paper introduces a fault-tolerant multi-agent reinforcement learning approach for cooperative connected and automated vehicles during highway on-ramp merging, enhancing safety and efficiency under observation faults.

Contribution

It proposes a novel MARL framework with adversarial fault injection and self-diagnosis for fault mitigation in CAVs during highway merging.

Findings

Significantly improves safety and efficiency under observation faults.

Outperforms baseline MARL methods in simulated highway merging.

Achieves near-fault-free performance in experiments.

Abstract

Multi-Agent Reinforcement Learning (MARL) holds significant promise for enabling cooperative driving among Connected and Automated Vehicles (CAVs). However, its practical application is hindered by a critical limitation, i.e., insufficient fault tolerance against observational faults. Such faults, which appear as perturbations in the vehicles' perceived data, can substantially compromise the performance of MARL-based driving systems. Addressing this problem presents two primary challenges. One is to generate adversarial perturbations that effectively stress the policy during training, and the other is to equip vehicles with the capability to mitigate the impact of corrupted observations. To overcome the challenges, we propose a fault-tolerant MARL method for cooperative on-ramp vehicles incorporating two key agents. First, an adversarial fault injection agent is co-trained to generate perturbations that actively challenge and harden the vehicle policies. Second, we design a novel fault-tolerant vehicle agent equipped with a self-diagnosis capability, which leverages the inherent spatio-temporal correlations in vehicle state sequences to detect faults and reconstruct credible observations, thereby shielding the policy from misleading inputs. Experiments in a simulated highway merging scenario demonstrate that our method significantly outperforms baseline MARL approaches, achieving near-fault-free levels of safety and efficiency under various observation fault patterns.