Anti-Jerk On-Ramp Merging Using Deep Reinforcement Learning

TL;DR

This paper employs Deep Reinforcement Learning with DDPG to develop a decentralized control strategy for high-speed on-ramp merging, effectively reducing jerk by 73% without compromising safety.

Contribution

It introduces a multi-objective reward function balancing collision avoidance and passenger comfort, demonstrating significant jerk reduction in on-ramp merging scenarios.

Findings

73% reduction in vehicle jerk with safety maintained

Decision strategies include merging ahead or behind main-road vehicles

Multi-objective reward function effectively balances safety and comfort

Abstract

Deep Reinforcement Learning (DRL) is used here for decentralized decision-making and longitudinal control for high-speed on-ramp merging. The DRL environment state includes the states of five vehicles: the merging vehicle, along with two preceding and two following vehicles when the merging vehicle is or is projected on the main road. The control action is the acceleration of the merging vehicle. Deep Deterministic Policy Gradient (DDPG) is the DRL algorithm for training to output continuous control actions. We investigated the relationship between collision avoidance for safety and jerk minimization for passenger comfort in the multi-objective reward function by obtaining the Pareto front. We found that, with a small jerk penalty in the multi-objective reward function, the vehicle jerk could be reduced by 73% compared with no jerk penalty while the collision rate was maintained at zero. Regardless of the jerk penalty, the merging vehicle exhibited decision-making strategies such as merging ahead or behind a main-road vehicle.