Deep Reinforcement Learning Aided Platoon Control Relying on V2X Information

TL;DR

This paper explores how Vehicle-to-Everything (V2X) communications enhance deep reinforcement learning-based platoon control by identifying optimal information sharing strategies to balance performance gains and complexity.

Contribution

It introduces SSDP models for platoon control with different information topologies and establishes theorems to compare their optimal policies, optimizing V2X information sharing.

Findings

V2X information improves platoon control performance.

Conditional KL divergence quantifies the value of shared information.

Optimal information sharing balances performance and complexity.

Abstract

The impact of Vehicle-to-Everything (V2X) communications on platoon control performance is investigated. Platoon control is essentially a sequential stochastic decision problem (SSDP), which can be solved by Deep Reinforcement Learning (DRL) to deal with both the control constraints and uncertainty in the platoon leading vehicle's behavior. In this context, the value of V2X communications for DRL-based platoon controllers is studied with an emphasis on the tradeoff between the gain of including exogenous information in the system state for reducing uncertainty and the performance erosion due to the curse-of-dimensionality. Our objective is to find the specific set of information that should be shared among the vehicles for the construction of the most appropriate state space. SSDP models are conceived for platoon control under different information topologies (IFT) by taking into account `just sufficient' information. Furthermore, theorems are established for comparing the performance of their optimal policies. In order to determine whether a piece of information should or should not be transmitted for improving the DRL-based control policy, we quantify its value by deriving the conditional KL divergence of the transition models. More meritorious information is given higher priority in transmission, since including it in the state space has a higher probability in offsetting the negative effect of having higher state dimensions. Finally, simulation results are provided to illustrate the theoretical analysis.