Reinforcement Learning-based Sequential Route Recommendation for System-Optimal Traffic Assignment

TL;DR

This paper introduces a reinforcement learning framework that sequentially recommends routes to travelers, aiming to achieve system-optimal traffic assignment by minimizing total travel time, and demonstrates its effectiveness on standard network models.

Contribution

It reformulates static traffic assignment as a deep RL problem with an MSA-guided algorithm, enabling convergence to system-optimal solutions in complex networks.

Findings

RL agent converges to SO solution in Braess network.

Achieves 0.35% deviation from SO in OW network.

Route set design impacts learning speed and quality.

Abstract

Modern navigation systems and shared mobility platforms increasingly rely on personalized route recommendations to improve individual travel experience and operational efficiency. However, a key question remains: can such sequential, personalized routing decisions collectively lead to system-optimal (SO) traffic assignment? This paper addresses this question by proposing a learning-based framework that reformulates the static SO traffic assignment problem as a single-agent deep reinforcement learning (RL) task. A central agent sequentially recommends routes to travelers as origin-destination (OD) demands arrive, to minimize total system travel time. To enhance learning efficiency and solution quality, we develop an MSA-guided deep Q-learning algorithm that integrates the iterative structure of traditional traffic assignment methods into the RL training process. The proposed approach is evaluated on both the Braess and Ortuzar-Willumsen (OW) networks. Results show that the RL agent converges to the theoretical SO solution in the Braess network and achieves only a 0.35% deviation in the OW network. Further ablation studies demonstrate that the route action set's design significantly impacts convergence speed and final performance, with SO-informed route sets leading to faster learning and better outcomes. This work provides a theoretically grounded and practically relevant approach to bridging individual routing behavior with system-level efficiency through learning-based sequential assignment.