Short Quantum Circuits in Reinforcement Learning Policies for the Vehicle Routing Problem

TL;DR

This paper introduces a hybrid classical-quantum approach using short quantum circuits within reinforcement learning policies for vehicle routing, demonstrating competitive performance and potential for scalable quantum integration.

Contribution

The paper presents a novel method replacing classical attention mechanisms with simple quantum circuits in reinforcement learning models for routing problems.

Findings

Quantum circuits can replace classical attention heads without performance loss.

The hybrid model performs competitively on vehicle routing problems.

The approach serves as a prototype for scalable quantum reinforcement learning.

Abstract

Quantum computing and machine learning have potential for symbiosis. However, in addition to the hardware limitations from current devices, there are still basic issues that must be addressed before quantum circuits can usefully incorporate with current machine learning tasks. We report a new strategy for such an integration in the context of attention models used for reinforcement learning. Agents that implement attention mechanisms have successfully been applied to certain cases of combinatorial routing problems by first encoding nodes on a graph and then sequentially decoding nodes until a route is selected. We demonstrate that simple quantum circuits can used in place of classical attention head layers while maintaining performance. Our method modifies the networks used in [1] by replacing key and query vectors for every node with quantum states that are entangled before being measured. The resulting hybrid classical-quantum agent is tested in the context of vehicle routing problems where its performance is competitive with the original classical approach. We regard our model as a prototype that can be scaled up and as an avenue for further study on the role of quantum computing in reinforcement learning.