Adaptive Tuning of Parameterized Traffic Controllers via Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a multi-agent reinforcement learning framework that adaptively tunes traffic controller parameters, improving robustness and efficiency in managing complex, time-varying transportation networks.

Contribution

It presents a novel multi-agent RL approach for adaptive parameter tuning of traffic controllers, enhancing resilience and training efficiency over traditional methods.

Findings

Multi-agent RL outperforms no-control and fixed-parameter controllers.

The framework matches single-agent RL performance with greater robustness.

Adaptive tuning improves traffic management under varying conditions.

Abstract

Effective traffic control is essential for mitigating congestion in transportation networks. Conventional traffic management strategies, including route guidance and ramp metering, often rely on state feedback controllers, which are used for their simplicity and reactivity; however, they lack the adaptability required to cope with complex and time-varying traffic dynamics. This paper proposes a multi-agent reinforcement learning (RL) framework in which each agent adaptively tunes the parameters of a state feedback traffic controller, combining the reactivity of state feedback controllers with the adaptability of RL. By tuning parameters at a lower frequency rather than directly determining control inputs at a high frequency, the RL agents achieve improved training efficiency while maintaining adaptability to varying traffic conditions. The multi-agent structure further enhances system robustness, as local controllers can operate independently in the event of partial failures. The proposed framework is evaluated on a simulated multi-class transportation network under varying traffic conditions. Results show that the proposed multi-agent framework outperforms the no-control and fixed-parameter state feedback control cases, while performing on par with the single-agent RL-based adaptive state feedback control, but with much greater resilience to disturbances.