Co-Optimization of On-Ramp Merging and Plug-In Hybrid Electric Vehicle Power Split Using Deep Reinforcement Learning

TL;DR

This paper demonstrates that deep reinforcement learning can effectively co-optimize on-ramp merging and energy management in a hybrid vehicle, ensuring collision-free merging while considering vehicle dynamics and powertrain constraints.

Contribution

It introduces a novel DRL-based co-optimization framework for on-ramp merging and PHEV energy management, integrating vehicle dynamics and powertrain constraints.

Findings

DRL successfully achieves collision-free merging.

Co-optimization leads to more economical merging.

Sequential approaches risk collisions due to neglecting powertrain limits.

Abstract

Current research on Deep Reinforcement Learning (DRL) for automated on-ramp merging neglects vehicle powertrain and dynamics. This work considers automated on-ramp merging for a power-split Plug-In Hybrid Electric Vehicle (PHEV), the 2015 Toyota Prius Plug-In, using DRL. The on-ramp merging control and the PHEV energy management are co-optimized such that the DRL policy directly outputs the power split between the engine and the electric motor. The testing results show that DRL can be successfully used for co-optimization, leading to collision-free on-ramp merging. When compared with sequential approaches wherein the upper-level on-ramp merging control and the lower-level PHEV energy management are performed independently and in sequence, we found that co-optimization results in economic but jerky on-ramp merging while sequential approaches may result in collisions due to neglecting powertrain power limit constraints in designing the upper-level on-ramp merging controller.