Bi-level RL-Heuristic Optimization for Real-world Winter Road Maintenance

TL;DR

This paper introduces a scalable bi-level reinforcement learning framework for large-scale winter road maintenance routing, improving efficiency, reducing emissions, and balancing workloads on real UK road networks.

Contribution

It presents a novel AI-driven bi-level optimization approach that effectively manages large real-world road networks for winter maintenance planning.

Findings

Reduced maximum vehicle travel time below two hours

Lowered carbon emissions in routing solutions

Achieved significant cost savings and workload balance

Abstract

Winter road maintenance is critical for ensuring public safety and reducing environmental impacts, yet existing methods struggle to manage large-scale routing problems effectively and mostly reply on human decision. This study presents a novel, scalable bi-level optimization framework, validated on real operational data on UK strategic road networks (M25, M6, A1), including interconnected local road networks in surrounding areas for vehicle traversing, as part of the highway operator's efforts to solve existing planning challenges. At the upper level, a reinforcement learning (RL) agent strategically partitions the road network into manageable clusters and optimally allocates resources from multiple depots. At the lower level, a multi-objective vehicle routing problem (VRP) is solved within each cluster, minimizing the maximum vehicle travel time and total carbon emissions. Unlike existing approaches, our method handles large-scale, real-world networks efficiently, explicitly incorporating vehicle-specific constraints, depot capacities, and road segment requirements. Results demonstrate significant improvements, including balanced workloads, reduced maximum travel times below the targeted two-hour threshold, lower emissions, and substantial cost savings. This study illustrates how advanced AI-driven bi-level optimization can directly enhance operational decision-making in real-world transportation and logistics.