Robust Single-Agent Reinforcement Learning for Regional Traffic Signal Control Under Demand Fluctuations

TL;DR

This paper presents a robust single-agent reinforcement learning framework for regional traffic signal control that adapts effectively to demand fluctuations, reducing congestion without complex multi-agent coordination.

Contribution

It introduces a novel RL-based TSC model using a centralized approach with a world model, improving robustness to demand fluctuations and simplifying coordination.

Findings

Significantly reduces queue lengths under demand fluctuations

Demonstrates robustness in multi-level OD demand scenarios

Outperforms traditional control methods in simulation

Abstract

Traffic congestion, primarily driven by intersection queuing, significantly impacts urban living standards, safety, environmental quality, and economic efficiency. While Traffic Signal Control (TSC) systems hold potential for congestion mitigation, traditional optimization models often fail to capture real-world traffic complexity and dynamics. This study introduces a novel single-agent reinforcement learning (RL) framework for regional adaptive TSC, circumventing the coordination complexities inherent in multi-agent systems through a centralized decision-making paradigm. The model employs an adjacency matrix to unify the encoding of road network topology, real-time queue states derived from probe vehicle data, and current signal timing parameters. Leveraging the efficient learning capabilities of the DreamerV3 world model, the agent learns control policies where actions sequentially select intersections and adjust their signal phase splits to regulate traffic inflow/outflow, analogous to a feedback control system. Reward design prioritizes queue dissipation, directly linking congestion metrics (queue length) to control actions. Simulation experiments conducted in SUMO demonstrate the model's effectiveness: under inference scenarios with multi-level (10%, 20%, 30%) Origin-Destination (OD) demand fluctuations, the framework exhibits robust anti-fluctuation capability and significantly reduces queue lengths. This work establishes a new paradigm for intelligent traffic control compatible with probe vehicle technology. Future research will focus on enhancing practical applicability by incorporating stochastic OD demand fluctuations during training and exploring regional optimization mechanisms for contingency events.