Novel Actor-Critic Algorithm for Robust Decision Making of CAV under Delays and Loss of V2X Data

TL;DR

This paper introduces a novel 'Blind Actor-Critic' algorithm designed to enable autonomous vehicles to maintain robust decision-making despite delays and data loss in V2X communication, improving safety and reliability.

Contribution

The paper presents a new actor-critic algorithm that compensates for V2X data delays and loss, enhancing autonomous vehicle control robustness in unreliable communication environments.

Findings

Improved training metrics over conventional algorithms.

Robust control performance under low V2X reliability.

Effective handling of temporal aperiodicity in data.

Abstract

Current autonomous driving systems heavily rely on V2X communication data to enhance situational awareness and the cooperation between vehicles. However, a major challenge when using V2X data is that it may not be available periodically because of unpredictable delays and data loss during wireless transmission between road stations and the receiver vehicle. This issue should be considered when designing control strategies for connected and autonomous vehicles. Therefore, this paper proposes a novel 'Blind Actor-Critic' algorithm that guarantees robust driving performance in V2X environment with delayed and/or lost data. The novel algorithm incorporates three key mechanisms: a virtual fixed sampling period, a combination of Temporal-Difference and Monte Carlo learning, and a numerical approximation of immediate reward values. To address the temporal aperiodicity problem of V2X data, we first illustrate this challenge. Then, we provide a detailed explanation of the Blind Actor-Critic algorithm where we highlight the proposed components to compensate for the temporal aperiodicity problem of V2X data. We evaluate the performance of our algorithm in a simulation environment and compare it to benchmark approaches. The results demonstrate that training metrics are improved compared to conventional actor-critic algorithms. Additionally, testing results show that our approach provides robust control, even under low V2X network reliability levels.