Traffic Light Control with Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning approach for real-time traffic light control that adapts to traffic conditions, significantly reducing congestion and travel times in urban intersections.

Contribution

It presents a novel deep Q learning framework with phase gating and memory mechanisms for dynamic traffic signal management, validated on real-world data.

Findings

Vehicle waiting time reduced by up to 100%

Queue lengths decreased significantly

Total travel time improved substantially

Abstract

Traffic light control is important for reducing congestion in urban mobility systems. This paper proposes a real-time traffic light control method using deep Q learning. Our approach incorporates a reward function considering queue lengths, delays, travel time, and throughput. The model dynamically decides phase changes based on current traffic conditions. The training of the deep Q network involves an offline stage from pre-generated data with fixed schedules and an online stage using real-time traffic data. A deep Q network structure with a "phase gate" component is used to simplify the model's learning task under different phases. A "memory palace" mechanism is used to address sample imbalance during the training process. We validate our approach using both synthetic and real-world traffic flow data on a road intersecting in Hangzhou, China. Results demonstrate significant performance improvements of the proposed method in reducing vehicle waiting time (57.1% to 100%), queue lengths (40.9% to 100%), and total travel time (16.8% to 68.0%) compared to traditional fixed signal plans.