Single-agent Reinforcement Learning Model for Regional Adaptive Traffic Signal Control

TL;DR

This paper introduces a single-agent reinforcement learning model for regional adaptive traffic signal control that leverages probe vehicle data for scalable and effective congestion management across multiple intersections.

Contribution

The study proposes a novel single-agent RL framework for regional ATSC that utilizes probe vehicle data, addressing scalability issues inherent in multi-agent approaches.

Findings

Effectively reduces regional congestion in simulations

Utilizes probe vehicle data for reliable state estimation

Demonstrates coordinated control across multiple intersections

Abstract

Several studies have employed reinforcement learning (RL) to address the challenges of regional adaptive traffic signal control (ATSC) and achieved promising results. In this field, existing research predominantly adopts multi-agent frameworks. However, the adoption of multi-agent frameworks presents challenges for scalability. Instead, the Traffic signal control (TSC) problem necessitates a single-agent framework. TSC inherently relies on centralized management by a single control center, which can monitor traffic conditions across all roads in the study area and coordinate the control of all intersections. This work proposes a single-agent RL-based regional ATSC model compatible with probe vehicle technology. Key components of the RL design include state, action, and reward function definitions. To facilitate learning and manage congestion, both state and reward functions are defined based on queue length, with action designed to regulate queue dynamics. The queue length definition used in this study differs slightly from conventional definitions but is closely correlated with congestion states. More importantly, it allows for reliable estimation using link travel time data from probe vehicles. With probe vehicle data already covering most urban roads, this feature enhances the proposed method's potential for widespread deployment. The method was comprehensively evaluated using the SUMO simulation platform. Experimental results demonstrate that the proposed model effectively mitigates large-scale regional congestion levels via coordinated multi-intersection control.