Reinforcement Learning for Solving the Vehicle Routing Problem

TL;DR

This paper introduces a reinforcement learning framework that trains a single model to solve various Vehicle Routing Problem instances efficiently, outperforming classical methods in solution quality after training.

Contribution

The paper presents a novel RL-based approach that learns a stochastic policy for VRP, eliminating the need for re-training on new instances and extending to problem variants.

Findings

Outperforms classical heuristics and OR-Tools on medium-sized VRP instances

Handles VRP variants like split delivery effectively

Produces near-optimal solutions with real-time inference

Abstract

We present an end-to-end framework for solving the Vehicle Routing Problem (VRP) using reinforcement learning. In this approach, we train a single model that finds near-optimal solutions for problem instances sampled from a given distribution, only by observing the reward signals and following feasibility rules. Our model represents a parameterized stochastic policy, and by applying a policy gradient algorithm to optimize its parameters, the trained model produces the solution as a sequence of consecutive actions in real time, without the need to re-train for every new problem instance. On capacitated VRP, our approach outperforms classical heuristics and Google's OR-Tools on medium-sized instances in solution quality with comparable computation time (after training). We demonstrate how our approach can handle problems with split delivery and explore the effect of such deliveries on the solution quality. Our proposed framework can be applied to other variants of the VRP such as the stochastic VRP, and has the potential to be applied more generally to combinatorial optimization problems.