The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality

TL;DR

This paper reviews the challenges faced in deploying reinforcement learning-based traffic signal control systems in real-world scenarios, emphasizing the need for addressing uncertainty, communication reliability, compliance, and heterogeneity among road users.

Contribution

It provides the first comprehensive review of challenges for RL-based TSC deployment and highlights areas needing further research with a systems thinking approach.

Findings

Progress has been made in addressing key challenges.

RL-based TSC systems face significant deployment hurdles.

More integrated research is needed for real-world implementation.

Abstract

Traffic signal control (TSC) is a high-stakes domain that is growing in importance as traffic volume grows globally. An increasing number of works are applying reinforcement learning (RL) to TSC; RL can draw on an abundance of traffic data to improve signalling efficiency. However, RL-based signal controllers have never been deployed. In this work, we provide the first review of challenges that must be addressed before RL can be deployed for TSC. We focus on four challenges involving (1) uncertainty in detection, (2) reliability of communications, (3) compliance and interpretability, and (4) heterogeneous road users. We show that the literature on RL-based TSC has made some progress towards addressing each challenge. However, more work should take a systems thinking approach that considers the impacts of other pipeline components on RL.