MARL-FWC: Optimal Coordination of Freeway Traffic Control Measures

TL;DR

This paper introduces MARL-FWC, a multi-agent reinforcement learning system for optimizing freeway traffic flow through coordinated ramp metering and dynamic speed limits, demonstrating significant improvements in travel time and speed.

Contribution

It proposes a novel collaborative multi-agent reinforcement learning framework with a microscopic network-level model and various control architectures for freeway traffic management.

Findings

Significant reduction in total travel time.

Increase in average traffic speed.

Effective coordination of control measures in simulations.

Abstract

The objective of this article is to optimize the overall traffic flow on freeways using multiple ramp metering controls plus its complementary Dynamic Speed Limits (DSLs). An optimal freeway operation can be reached when minimizing the difference between the freeway density and the critical ratio for maximum traffic flow. In this article, a Multi-Agent Reinforcement Learning for Freeways Control (MARL-FWC) system for ramps metering and DSLs is proposed. MARL-FWC introduces a new microscopic framework at the network level based on collaborative Markov Decision Process modeling (Markov game) and an associated cooperative Q-learning algorithm. The technique incorporates payoff propagation (Max-Plus algorithm) under the coordination graphs framework, particularly suited for optimal control purposes. MARL-FWC provides three control designs: fully independent, fully distributed, and centralized; suited for different network architectures. MARL-FWC was extensively tested in order to assess the proposed model of the joint payoff, as well as the global payoff. Experiments are conducted with heavy traffic flow under the renowned VISSIM traffic simulator to evaluate MARL-FWC. The experimental results show a significant decrease in the total travel time and an increase in the average speed (when compared with the base case) while maintaining an optimal traffic flow.