Quantum Reinforcement Learning with Transformers for the Capacitated Vehicle Routing Problem

TL;DR

This study compares classical, quantum, and hybrid reinforcement learning models with transformer architectures for solving the Capacitated Vehicle Routing Problem, demonstrating quantum advantages in robustness and solution quality.

Contribution

It introduces a hybrid quantum-classical RL approach with transformer architectures for CVRP, showing improved performance over classical methods.

Findings

Quantum models outperform classical in routing quality.

Hybrid architecture achieves the best overall performance.

Quantum-based models produce more structured routes.

Abstract

This paper addresses the Capacitated Vehicle Routing Problem (CVRP) by comparing classical and quantum Reinforcement Learning (RL) approaches. An Advantage Actor-Critic (A2C) agent is implemented in classical, full quantum, and hybrid variants, integrating transformer architectures to capture the relationships between vehicles, clients, and the depot through self- and cross-attention mechanisms. The experiments focus on multi-vehicle scenarios with capacity constraints, considering 20 clients and 4 vehicles, and are conducted over ten independent runs. Performance is assessed using routing distance, route compactness, and route overlap. The results show that all three approaches are capable of learning effective routing policies. However, quantum-enhanced models outperform the classical baseline and produce more robust route organization, with the hybrid architecture achieving the best overall performance across distance, compactness, and route overlap. In addition to quantitative improvements, qualitative visualizations reveal that quantum-based models generate more structured and coherent routing solutions. These findings highlight the potential of hybrid quantum-classical reinforcement learning models for addressing complex combinatorial optimization problems such as the CVRP.