Systematic Analyses of Reinforcement Learning Controllers in Signalized Urban Corridors

TL;DR

This paper systematically compares different reinforcement learning controllers and classical methods for urban traffic corridors, revealing insights into their capacity and self-organizing traffic patterns.

Contribution

It introduces a comprehensive analysis of RL controllers in multi-junction traffic networks and demonstrates the potential for decentralized controllers to promote self-organized traffic flow.

Findings

Decentralized RL controllers achieve comparable capacity regions to centralized ones.

Parameter-sharing RL controllers can generalize to larger networks.

Traffic may self-organize into green waves without explicit coordination.

Abstract

In this work, we extend our systematic capacity region perspective to multi-junction traffic networks, focussing on the special case of an urban corridor network. In particular, we train and evaluate centralized, fully decentralized, and parameter-sharing decentralized RL controllers, and compare their capacity regions and ATTs together with a classical baseline MaxPressure controller. Further, we show how the parametersharing controller may be generalised to be deployed on a larger network than it was originally trained on. In this setting, we show some initial findings that suggest that even though the junctions are not formally coordinated, traffic may self organise into `green waves'.