Control of a Mixed Autonomy Signalised Urban Intersection: An Action-Delayed Reinforcement Learning Approach

TL;DR

This paper introduces an action-delayed reinforcement learning method for traffic signal control at urban intersections with mixed autonomy, aiming to reduce vehicle queues and energy costs by enabling autonomous vehicles to adapt to traffic light signals.

Contribution

It presents a novel reinforcement learning approach for mixed autonomy traffic control that accounts for delayed actions and improves vehicle flow and energy efficiency.

Findings

Algorithm converges empirically.

Reduces energy costs of autonomous vehicles.

Improves traffic flow at intersections.

Abstract

We consider a mixed autonomy scenario where the traffic intersection controller decides whether the traffic light will be green or red at each lane for multiple traffic-light blocks. The objective of the traffic intersection controller is to minimize the queue length at each lane and maximize the outflow of vehicles over each block. We consider that the traffic intersection controller informs the autonomous vehicle (AV) whether the traffic light will be green or red for the future traffic-light block. Thus, the AV can adapt its dynamics by solving an optimal control problem. We model the decision process of the traffic intersection controller as a deterministic delay Markov decision process owing to the delayed action by the traffic controller. We propose Reinforcement-learning based algorithm to obtain the optimal policy. We show - empirically - that our algorithm converges and reduces the energy costs of AVs drastically as the traffic controller communicates with the AVs.