Safe Reinforcement Learning-Based Eco-Driving Control for Mixed Traffic Flows With Disturbances

TL;DR

This paper introduces a safe, learning-based eco-driving control framework that combines reinforcement learning with tube-based MPC to optimize energy efficiency and ensure safety in mixed traffic with disturbances.

Contribution

It proposes a novel framework integrating RL and RMPC to handle uncertainties and safety constraints in eco-driving for mixed traffic environments.

Findings

10.88% average improvement in energy efficiency

Prevents inter-vehicle collisions effectively

Enhances control robustness under disturbances

Abstract

This paper presents a safe learning-based eco-driving framework tailored for mixed traffic flows, which aims to optimize energy efficiency while guaranteeing safety during real-system operations. Even though reinforcement learning (RL) is capable of optimizing energy efficiency in intricate environments, it is challenged by safety requirements during the training process. The lack of safety guarantees is the other concern when deploying a trained policy in real-world application. Compared with RL, model predicted control (MPC) can handle constrained dynamics systems, ensuring safe driving. However, the major challenges lie in complicated eco-driving tasks and the presence of disturbances, which respectively challenge the MPC design and the satisfaction of constraints. To address these limitations, the proposed framework incorporates the tube-based enhanced MPC (RMPC) to ensure the safe execution of the RL policy under disturbances, thereby improving the control robustness. RL not only optimizes the energy efficiency of the connected and automated vehicle in mixed traffic but also handles more uncertain scenarios, in which the energy consumption of the human-driven vehicle and its diverse and stochastic driving behaviors are considered in the optimization framework. Simulation results demonstrate that the proposed algorithm, compared with RMPC technique, shows an average improvement of 10.88% in holistic energy efficiency, while compared with RL algorithm, it effectively prevents inter-vehicle collisions.