Hybridising Reinforcement Learning and Heuristics for Hierarchical Directed Arc Routing Problems

TL;DR

This paper introduces HRDA, a hybrid reinforcement learning and heuristic algorithm, to efficiently solve hierarchical directed arc routing problems, significantly improving speed while maintaining solution quality.

Contribution

It presents a novel hybrid RL-heuristic approach for HDCARP, addressing computational challenges and enhancing solution efficiency for large-scale instances.

Findings

Significant speed improvements over existing methods

Maintains high solution quality

Effective dynamic adaptation to problem changes

Abstract

The Hierarchical Directed Capacitated Arc Routing Problem (HDCARP) is an extension of the Capacitated Arc Routing Problem (CARP), where the arcs of a graph are divided into classes based on their priority. The traversal of these classes is determined by either precedence constraints or a hierarchical objective, resulting in two distinct HDCARP variants. To the best of our knowledge, only one matheuristic has been proposed for these variants, but it performs relatively slowly, particularly for large-scale instances (Ha et al., 2024). In this paper, we propose a fast heuristic to efficiently address the computational challenges of HDCARP. Furthermore, we incorporate Reinforcement Learning (RL) into our heuristic to effectively guide the selection of local search operators, resulting in a hybrid algorithm. We name this hybrid algorithm as the Hybrid Reinforcement Learning and Heuristic Algorithm for Directed Arc Routing (HRDA). The hybrid algorithm adapts to changes in the problem dynamically, using real-time feedback to improve routing strategies and solution's quality by integrating heuristic methods. Extensive computational experiments on artificial instances demonstrate that this hybrid approach significantly improves the speed of the heuristic without deteriorating the solution quality. Our source code is publicly available at: https://github.com/HySonLab/ArcRoute