Optimizing Traffic Signal Control using High-Dimensional State Representation and Efficient Deep Reinforcement Learning

TL;DR

This paper demonstrates that high-dimensional state representations in reinforcement learning for traffic signal control, enabled by vehicle-to-infrastructure communication, can significantly improve performance and can be made computationally efficient through model pruning.

Contribution

It shows that high-dimensional state representations can enhance RL-based traffic signal control, challenging previous assumptions, and introduces model compression techniques for efficiency.

Findings

High-dimensional states improve TSC performance by up to 17.9%.

V2I communication enables cost-effective high-dimensional state data.

Model pruning enhances computational efficiency for large state models.

Abstract

In reinforcement learning-based (RL-based) traffic signal control (TSC), decisions on the signal timing are made based on the available information on vehicles at a road intersection. This forms the state representation for the RL environment which can either be high-dimensional containing several variables or a low-dimensional vector. Current studies suggest that using high dimensional state representations does not lead to improved performance on TSC. However, we argue, with experimental results, that the use of high dimensional state representations can, in fact, lead to improved TSC performance with improvements up to 17.9% of the average waiting time. This high-dimensional representation is obtainable using the cost-effective vehicle-to-infrastructure (V2I) communication, encouraging its adoption for TSC. Additionally, given the large size of the state, we identified the need to have computational efficient models and explored model compression via pruning.