# AlphaRouter: Bridging the Gap Between Reinforcement Learning and Optimization for Vehicle Routing with Monte Carlo Tree Searches

## TL;DR

AlphaRouter combines reinforcement learning and optimization to solve vehicle routing problems more effectively than traditional methods.

## Contribution

AlphaRouter bridges reinforcement learning and optimization using Monte Carlo tree search for routing problems.

## Key findings

- AlphaRouter outperforms original reinforcement learning approaches in solution quality.
- The method achieves performance comparable to classical heuristics for routing problems.
- Attention-enabled networks improve routing decisions in the proposed framework.

## Abstract

Deep reinforcement learning (DRL) as a routing problem solver has shown promising results in recent studies. However, an inherent gap exists between computationally driven DRL and optimization-based heuristics. While a DRL algorithm for a certain problem is able to solve several similar problem instances, traditional optimization algorithms focus on optimizing solutions to one specific problem instance. In this paper, we propose an approach, AlphaRouter, which solves routing problems while bridging the gap between reinforcement learning and optimization. Fitting to routing problems, our approach first proposes attention-enabled policy and value networks consisting of a policy network that produces a probability distribution over all possible nodes and a value network that produces the expected distance from any given state. We modify a Monte Carlo tree search (MCTS) for the routing problems, selectively combining it with the routing problems. Our experiments demonstrate that the combined approach is promising and yields better solutions compared to original reinforcement learning (RL) approaches without MCTS, with good performance comparable to classical heuristics.