Scalable and Reliable Multi-agent Reinforcement Learning for Traffic Assignment

TL;DR

This paper introduces MARL-OD-DA, a scalable and reliable multi-agent reinforcement learning framework for large-scale traffic assignment, outperforming existing methods in efficiency and solution quality on city-level networks.

Contribution

The study presents a novel MARL framework that redefines agents as OD pair routers and employs a Dirichlet-based action space to enhance scalability and reliability in traffic assignment.

Findings

Effectively handles medium-sized networks with extensive OD demand.

Achieves 94.99% lower relative gap in SiouxFalls network.

Surpasses existing MARL methods in solution quality and convergence speed.

Abstract

The evolution of metropolitan cities and the increase in travel demands impose stringent requirements on traffic assignment methods. Multi-agent reinforcement learning (MARL) approaches outperform traditional methods in modeling adaptive routing behavior without requiring explicit system dynamics, which is beneficial for real-world deployment. However, MARL frameworks face challenges in scalability and reliability when managing extensive networks with substantial travel demand, which limiting their practical applicability in solving large-scale traffic assignment problems. To address these challenges, this study introduces MARL-OD-DA, a new MARL framework for the traffic assignment problem, which redefines agents as origin-destination (OD) pair routers rather than individual travelers, significantly enhancing scalability. Additionally, a Dirichlet-based action space with action pruning and a reward function based on the local relative gap are designed to enhance solution reliability and improve convergence efficiency. Experiments demonstrate that the proposed MARL framework effectively handles medium-sized networks with extensive and varied city-level OD demand, surpassing existing MARL methods. When implemented in the SiouxFalls network, MARL-OD-DA achieves better assignment solutions in 10 steps, with a relative gap that is 94.99% lower than that of conventional methods.