Reinforcement Learning for Multi-Truck Vehicle Routing Problems

TL;DR

This paper introduces an advanced deep reinforcement learning approach with encoder-decoder models tailored for complex multi-truck vehicle routing problems involving multi-leg routes, demonstrating improved solutions in real-world supply chain scenarios.

Contribution

The paper develops new extensions to encoder-decoder attention models enabling RL to handle multi-truck, multi-leg VRPs, scalable from small to large supply chain problems.

Findings

Our models outperform previous solutions in a real automotive supply chain environment.

The approach effectively generalizes from small to large problem instances.

Deep RL can be adapted for complex, real-world vehicle routing challenges.

Abstract

Deep reinforcement learning (RL) has been shown to be effective in producing approximate solutions to some vehicle routing problems (VRPs), especially when using policies generated by encoder-decoder attention mechanisms. While these techniques have been quite successful for relatively simple problem instances, there are still under-researched and highly complex VRP variants for which no effective RL method has been demonstrated. In this work we focus on one such VRP variant, which contains multiple trucks and multi-leg routing requirements. In these problems, demand is required to move along sequences of nodes, instead of just from a start node to an end node. With the goal of making deep RL a viable strategy for real-world industrial-scale supply chain logistics, we develop new extensions to existing encoder-decoder attention models which allow them to handle multiple trucks and multi-leg routing requirements. Our models have the advantage that they can be trained for a small number of trucks and nodes, and then embedded into a large supply chain to yield solutions for larger numbers of trucks and nodes. We test our approach on a real supply chain environment arising in the operations of Japanese automotive parts manufacturer Aisin Corporation, and find that our algorithm outperforms Aisin's previous best solution.