Learning to Search for Vehicle Routing with Multiple Time Windows

TL;DR

This paper introduces a reinforcement learning-enhanced adaptive variable neighborhood search method for solving complex vehicle routing problems with multiple time windows, improving solution quality and efficiency.

Contribution

It develops a novel RL-based framework that dynamically guides neighborhood operator selection using real-time solution states and a transformer neural network.

Findings

RL-AVNS outperforms traditional VNS and AVNS in solution quality.

The method generalizes well to unseen problem instances.

Significant computational efficiency improvements are achieved.

Abstract

In this study, we propose a reinforcement learning-based adaptive variable neighborhood search (RL-AVNS) method designed for effectively solving the Vehicle Routing Problem with Multiple Time Windows (VRPMTW). Unlike traditional adaptive approaches that rely solely on historical operator performance, our method integrates a reinforcement learning framework to dynamically select neighborhood operators based on real-time solution states and learned experience. We introduce a fitness metric that quantifies customers' temporal flexibility to improve the shaking phase, and employ a transformer-based neural policy network to intelligently guide operator selection during the local search. Extensive computational experiments are conducted on realistic scenarios derived from the replenishment of unmanned vending machines, characterized by multiple clustered replenishment windows. Results demonstrate that RL-AVNS significantly outperforms traditional variable neighborhood search (VNS), adaptive VNS (AVNS), and state-of-the-art learning-based heuristics, achieving substantial improvements in solution quality and computational efficiency across various instance scales and time window complexities. Particularly notable is the algorithm's capability to generalize effectively to problem instances not encountered during training, underscoring its practical utility for complex logistics scenarios.