Antifragile perimeter control: Anticipating and gaining from disruptions with reinforcement learning

TL;DR

This paper introduces an antifragile reinforcement learning approach for traffic control that not only withstands disruptions but also improves performance, demonstrating significant gains and robustness in real-world scenarios.

Contribution

It develops a novel antifragile deep reinforcement learning algorithm incorporating traffic state derivatives and redundancy, enhancing urban traffic management under disruptions.

Findings

Achieves up to 41.9% performance improvement over baselines.

Demonstrates lower skewness indicating antifragility.

Effective under limited data observability.

Abstract

The optimal operation of transportation systems is often susceptible to unexpected disruptions. Many established control strategies reliant on mathematical models can struggle with real-world disruptions, leading to significant divergence from their anticipated efficiency. This study integrates the cutting-edge concept of antifragility with learning-based traffic control strategies to optimize urban road network operations under disruptions. Antifragile systems not only withstand and recover from stressors but also thrive and enhance performance in the presence of such adversarial events. Incorporating antifragile modules composed of traffic state derivatives and redundancy, a deep reinforcement learning algorithm is developed. Subsequently, it is evaluated in a cordon-shaped transportation network and a case study with real-world data. Promising results highlight that the proposed algorithm provides: (i) superior performance achieving up to 27.6% and 41.9% performance gain over baselines under increasing demand and supply disruptions, (ii) lower distribution skewness under disruptions, demonstrating its relative antifragility against baselines, (iii) effectiveness under limited observability due to real-world data availability constraints, and (iv) the robustness and transferability to be combined with various state-of-the-art RL frameworks. The proposed antifragile methodology is generalizable and holds potential for applications beyond traffic engineering, offering integration into control systems exposed to disruptions across various disciplines.