Learning an Interpretable Traffic Signal Control Policy

TL;DR

This paper introduces and evaluates interpretable polynomial control policies for traffic signals, demonstrating they can match neural network performance and significantly reduce vehicle delays through reinforcement learning.

Contribution

It proposes a method for training interpretable traffic signal policies using reinforcement learning, showing they are as effective as neural networks in optimizing traffic flow.

Findings

Interpretable polynomial policies can match neural network performance.

Deep Q-learning variants effectively train interpretable control functions.

Up to 19.4% reduction in vehicle delay with the proposed method.

Abstract

Signalized intersections are managed by controllers that assign right of way (green, yellow, and red lights) to non-conflicting directions. Optimizing the actuation policy of such controllers is expected to alleviate traffic congestion and its adverse impact. Given such a safety-critical domain, the affiliated actuation policy is required to be interpretable in a way that can be understood and regulated by a human. This paper presents and analyzes several on-line optimization techniques for tuning interpretable control functions. Although these techniques are defined in a general way, this paper assumes a specific class of interpretable control functions (polynomial functions) for analysis purposes. We show that such an interpretable policy function can be as effective as a deep neural network for approximating an optimized signal actuation policy. We present empirical evidence that supports the use of value-based reinforcement learning for on-line training of the control function. Specifically, we present and study three variants of the Deep Q-learning algorithm that allow the training of an interpretable policy function. Our Deep Regulatable Hardmax Q-learning variant is shown to be particularly effective in optimizing our interpretable actuation policy, resulting in up to 19.4% reduced vehicles delay compared to commonly deployed actuated signal controllers.