Dynamic Queue-Jump Lane for Emergency Vehicles under Partially Connected Settings: A Multi-Agent Deep Reinforcement Learning Approach

TL;DR

This paper introduces a multi-agent deep reinforcement learning approach to optimize dynamic queue jump lanes for emergency vehicles in urban traffic, improving response times under partial vehicle connectivity.

Contribution

It develops a novel Markov decision process and multi-agent reinforcement learning framework for real-time coordination of connected vehicles to enhance emergency vehicle passage.

Findings

Up to 30% reduction in emergency vehicle travel time.

Effective coordination strategies validated in SUMO micro-simulation.

Handles varying numbers of connected vehicles and driver behaviors.

Abstract

Emergency vehicle (EMV) service is a key function of cities and is exceedingly challenging due to urban traffic congestion. The main reason behind EMV service delay is the lack of communication and cooperation between vehicles blocking EMVs. In this paper, we study the improvement of EMV service under V2X connectivity. We consider the establishment of dynamic queue jump lanes (DQJLs) based on real-time coordination of connected vehicles in the presence of non-connected human-driven vehicles. We develop a novel Markov decision process formulation for the DQJL coordination strategies, which explicitly accounts for the uncertainty of drivers' yielding pattern to approaching EMVs. Based on pairs of neural networks representing actors and critics for agent vehicles, we develop a multi-agent actor-critic deep reinforcement learning algorithm that handles a varying number of vehicles and a random proportion of connected vehicles in the traffic. Approaching the optimal coordination strategies via indirect and direct reinforcement learning, we present two schemata to address multi-agent reinforcement learning on this connected vehicle application. Both approaches are validated, on a micro-simulation testbed SUMO, to establish a DQJL fast and safely. Validation results reveal that, with DQJL coordination strategies, it saves up to 30% time for EMVs to pass a link-level intelligent urban roadway than the baseline scenario.