Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning

TL;DR

This paper introduces a unified Lyapunov-based framework for adaptive traffic signal control, integrating pressure and flow methods, and enhances reinforcement learning with a new reward function to improve traffic management.

Contribution

It develops a Lyapunov control framework that unifies flow and pressure methods and designs a novel RL reward function for adaptive traffic signal control.

Findings

The proposed method outperforms traditional and RL-based controls in reducing vehicle waiting times.

The framework links back-pressure to differential queue lengths and saturation flows.

Numerical tests validate the effectiveness across various traffic scenarios.

Abstract

In traffic signal control, flow-based (optimizing the overall flow) and pressure-based methods (equalizing and alleviating congestion) are commonly used but often considered separately. This study introduces a unified framework using Lyapunov control theory, defining specific Lyapunov functions respectively for these methods. We have found interesting results. For example, the well-recognized back-pressure method is equal to differential queue lengths weighted by intersection lane saturation flows. We further improve it by adding basic traffic flow theory. Rather than ensuring that the control system be stable, the system should be also capable of adaptive to various performance metrics. Building on insights from Lyapunov theory, this study designs a reward function for the Reinforcement Learning (RL)-based network signal control, whose agent is trained with Double Deep Q-Network (DDQN) for effective control over complex traffic networks. The proposed algorithm is compared with several traditional and RL-based methods under pure passenger car flow and heterogenous traffic flow including freight, respectively. The numerical tests demonstrate that the proposed method outperforms the alternative control methods across different traffic scenarios, covering corridor and general network situations each with varying traffic demands, in terms of the average network vehicle waiting time per vehicle.