Using a Deep Reinforcement Learning Agent for Traffic Signal Control

TL;DR

This paper presents a deep reinforcement learning-based traffic signal control system that leverages high-quality data and a novel state encoding to significantly improve traffic flow metrics in a microsimulator.

Contribution

It introduces a new discrete traffic state encoding and applies deep Q-learning to develop an adaptive traffic signal control agent.

Findings

Reduces average cumulative delay by 82%

Decreases average queue length by 66%

Lowers average travel time by 20%

Abstract

Ensuring transportation systems are efficient is a priority for modern society. Technological advances have made it possible for transportation systems to collect large volumes of varied data on an unprecedented scale. We propose a traffic signal control system which takes advantage of this new, high quality data, with minimal abstraction compared to other proposed systems. We apply modern deep reinforcement learning methods to build a truly adaptive traffic signal control agent in the traffic microsimulator SUMO. We propose a new state space, the discrete traffic state encoding, which is information dense. The discrete traffic state encoding is used as input to a deep convolutional neural network, trained using Q-learning with experience replay. Our agent was compared against a one hidden layer neural network traffic signal control agent and reduces average cumulative delay by 82%, average queue length by 66% and average travel time by 20%.