Deep Reinforcement Learning for the Joint Control of Traffic Light Signaling and Vehicle Speed Advice

TL;DR

This paper introduces a novel deep reinforcement learning approach that jointly controls traffic lights and vehicle speed advice, significantly reducing vehicle delays and smoothing traffic flow in urban environments.

Contribution

It is the first to combine traffic light control and vehicle speed advice learning, demonstrating improved traffic efficiency over separate control methods.

Findings

Joint control reduces average vehicle trip delays in most scenarios.

Speed advice smooths vehicle velocity profiles near traffic lights.

Approach has potential to reduce congestion and environmental impact.

Abstract

Traffic congestion in dense urban centers presents an economical and environmental burden. In recent years, the availability of vehicle-to-anything communication allows for the transmission of detailed vehicle states to the infrastructure that can be used for intelligent traffic light control. The other way around, the infrastructure can provide vehicles with advice on driving behavior, such as appropriate velocities, which can improve the efficacy of the traffic system. Several research works applied deep reinforcement learning to either traffic light control or vehicle speed advice. In this work, we propose a first attempt to jointly learn the control of both. We show this to improve the efficacy of traffic systems. In our experiments, the joint control approach reduces average vehicle trip delays, w.r.t. controlling only traffic lights, in eight out of eleven benchmark scenarios. Analyzing the qualitative behavior of the vehicle speed advice policy, we observe that this is achieved by smoothing out the velocity profile of vehicles nearby a traffic light. Learning joint control of traffic signaling and speed advice in the real world could help to reduce congestion and mitigate the economical and environmental repercussions of today's traffic systems.