Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning

TL;DR

This paper introduces RAMRL, a novel multi-modal reinforcement learning approach that enhances the robustness and safety of autonomous on-ramp merging by integrating wireless communication, sensor data, and data augmentation techniques.

Contribution

The paper presents a new reinforcement learning framework that combines multi-modal observations and data augmentation to improve on-ramp merging for autonomous vehicles.

Findings

Effective in handling noisy sensor data

Improves safety and traffic efficiency

Demonstrated success in SUMO simulations

Abstract

Despite the success of AI-enabled onboard perception, on-ramp merging has been one of the main challenges for autonomous driving. Due to limited sensing range of onboard sensors, a merging vehicle can hardly observe main road conditions and merge properly. By leveraging the wireless communications between connected and automated vehicles (CAVs), a merging CAV has potential to proactively obtain the intentions of nearby vehicles. However, CAVs can be prone to inaccurate observations, such as the noisy basic safety messages (BSM) and poor quality surveillance images. In this paper, we present a novel approach for Robust on-ramp merge of CAVs via Augmented and Multi-modal Reinforcement Learning, named by RAMRL. Specifically, we formulate the on-ramp merging problem as a Markov decision process (MDP) by taking driving safety, comfort driving behavior, and traffic efficiency into account. To provide reliable merging maneuvers, we simultaneously leverage BSM and surveillance images for multi-modal observation, which is used to learn a policy model through proximal policy optimization (PPO). Moreover, to improve data efficiency and provide better generalization performance, we train the policy model with augmented data (e.g., noisy BSM and noisy surveillance images). Extensive experiments are conducted with Simulation of Urban MObility (SUMO) platform under two typical merging scenarios. Experimental results demonstrate the effectiveness and efficiency of our robust on-ramp merging design.