CoTV: Cooperative Control for Traffic Light Signals and Connected Autonomous Vehicles using Deep Reinforcement Learning

TL;DR

This paper introduces CoTV, a multi-agent deep reinforcement learning system that cooperatively controls traffic lights and connected autonomous vehicles to optimize travel time, fuel efficiency, emissions, and safety in urban traffic.

Contribution

CoTV is the first system to simultaneously optimize traffic lights and CAVs using multi-agent DRL, improving coordination and convergence in large-scale traffic scenarios.

Findings

CoTV reduces travel time, fuel consumption, and emissions in simulations.

The system demonstrates effective coordination in mixed-autonomy urban traffic.

CoTV converges efficiently in large-scale multi-agent scenarios.

Abstract

The target of reducing travel time only is insufficient to support the development of future smart transportation systems. To align with the United Nations Sustainable Development Goals (UN-SDG), a further reduction of fuel and emissions, improvements of traffic safety, and the ease of infrastructure deployment and maintenance should also be considered. Different from existing work focusing on the optimization of the control in either traffic light signal (to improve the intersection throughput), or vehicle speed (to stabilize the traffic), this paper presents a multi-agent Deep Reinforcement Learning (DRL) system called CoTV, which Cooperatively controls both Traffic light signals and Connected Autonomous Vehicles (CAV). Therefore, our CoTV can well balance the achievement of the reduction of travel time, fuel, and emissions. In the meantime, CoTV can also be easy to deploy by cooperating with only one CAV that is the nearest to the traffic light controller on each incoming road. This enables more efficient coordination between traffic light controllers and CAV, thus leading to the convergence of training CoTV under the large-scale multi-agent scenario that is traditionally difficult to converge. We give the detailed system design of CoTV and demonstrate its effectiveness in a simulation study using SUMO under various grid maps and realistic urban scenarios with mixed-autonomy traffic.