Intelligent Traffic Light via Policy-based Deep Reinforcement Learning

TL;DR

This paper presents a policy-based deep reinforcement learning approach using PPO to optimize traffic light control in smart cities, outperforming value-based methods and adapting to variable traffic conditions.

Contribution

It introduces the use of PPO for traffic light control, compares it with other RL methods, and demonstrates robustness and adaptability to unbalanced traffic flows.

Findings

PPO outperforms DQN and DDQN in traffic light control.

Variable time interval phases improve traffic flow.

The controller is robust to environment and action disturbances.

Abstract

Intelligent traffic lights in smart cities can optimally reduce traffic congestion. In this study, we employ reinforcement learning to train the control agent of a traffic light on a simulator of urban mobility. As a difference from existing works, a policy-based deep reinforcement learning method, Proximal Policy Optimization (PPO), is utilized other than value-based methods such as Deep Q Network (DQN) and Double DQN (DDQN). At first, the obtained optimal policy from PPO is compared to those from DQN and DDQN. It is found that the policy from PPO performs better than the others. Next, instead of the fixed-interval traffic light phases, we adopt the light phases with variable time intervals, which result in a better policy to pass the traffic flow. Then, the effects of environment and action disturbances are studied to demonstrate the learning-based controller is robust. At last, we consider unbalanced traffic flows and find that an intelligent traffic light can perform moderately well for the unbalanced traffic scenarios, although it learns the optimal policy from the balanced traffic scenarios only.