UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control

TL;DR

This paper introduces UniTSA, a universal reinforcement learning framework for traffic signal control in V2X environments, capable of generalizing across diverse intersection structures through novel agent design and state augmentation methods.

Contribution

The paper presents a novel RL framework with a junction matrix and traffic state augmentation, enabling effective traffic signal control across various intersection types.

Findings

Framework outperforms existing methods in diverse intersection scenarios

Enhanced generalization capability demonstrated through extensive experiments

Source code available for reproducibility and further research

Abstract

Traffic congestion is a persistent problem in urban areas, which calls for the development of effective traffic signal control (TSC) systems. While existing Reinforcement Learning (RL)-based methods have shown promising performance in optimizing TSC, it is challenging to generalize these methods across intersections of different structures. In this work, a universal RL-based TSC framework is proposed for Vehicle-to-Everything (V2X) environments. The proposed framework introduces a novel agent design that incorporates a junction matrix to characterize intersection states, making the proposed model applicable to diverse intersections. To equip the proposed RL-based framework with enhanced capability of handling various intersection structures, novel traffic state augmentation methods are tailor-made for signal light control systems. Finally, extensive experimental results derived from multiple intersection configurations confirm the effectiveness of the proposed framework. The source code in this work is available at https://github.com/wmn7/Universal_Light