Dynamic network congestion pricing based on deep reinforcement learning

TL;DR

This paper introduces a deep reinforcement learning-based dynamic congestion pricing method that employs a distributed, cooperative learning scheme to effectively reduce traffic congestion in large-scale urban road networks.

Contribution

It proposes a novel distributed and cooperative DRL approach with spatially shared rewards and switching learning, enabling efficient congestion management in complex networks.

Findings

Effective congestion reduction demonstrated in Sioux Falls Network

Novel distributed cooperative learning scheme improves scalability

Method achieves fast and computationally efficient learning

Abstract

Traffic congestion is a serious problem in urban areas. Dynamic congestion pricing is one of the useful schemes to eliminate traffic congestion in strategic scale. However, in the reality, an optimal dynamic congestion pricing is very difficult or impossible to determine theoretically, because road networks are usually large and complicated, and behavior of road users is uncertain. To account for this challenge, this work proposes a dynamic congestion pricing method using deep reinforcement learning (DRL). It is designed to eliminate traffic congestion based on observable data in general large-scale road networks, by leveraging the data-driven nature of deep reinforcement learning. One of the novel elements of the proposed method is the distributed and cooperative learning scheme. Specifically, the DRL is implemented by a spatial-temporally distributed manner, and cooperation among DRL agents is established by novel techniques we call spatially shared reward and temporally switching learning. It enables fast and computationally efficient learning in large-scale networks. The numerical experiments using Sioux Falls Network showed that the proposed method works well thanks to the novel learning scheme.