A Large Language Model-Enhanced Q-learning for Capacitated Vehicle Routing Problem with Time Windows

TL;DR

This paper introduces a novel LLM-enhanced Q-learning framework for solving the NP-hard CVRPTW, combining adaptive training, self-correction, and prioritized experience replay to improve solution quality and training efficiency.

Contribution

It presents a new LLM-guided Q-learning approach with a two-phase training and a self-correction mechanism for CVRPTW, advancing solution methods for complex routing problems.

Findings

Achieves 7.3% average cost reduction over traditional Q-learning.

Requires fewer training steps for convergence.

Demonstrates effectiveness in real-time emergency constraints.

Abstract

The Capacitated Vehicle Routing Problem with Time Windows (CVRPTW) is a classic NP-hard combinatorial optimization problem widely applied in logistics distribution and transportation management. Its complexity stems from the constraints of vehicle capacity and time windows, which pose significant challenges to traditional approaches. Advances in Large Language Models (LLMs) provide new possibilities for finding approximate solutions to CVRPTW. This paper proposes a novel LLM-enhanced Q-learning framework to address the CVRPTW with real-time emergency constraints. Our solution introduces an adaptive two-phase training mechanism that transitions from the LLM-guided exploration phase to the autonomous optimization phase of Q-network. To ensure reliability, we design a three-tier self-correction mechanism based on the Chain-of-Thought (CoT) for LLMs: syntactic validation, semantic verification, and physical constraint enforcement. In addition, we also prioritized replay of the experience generated by LLMs to amplify the regulatory role of LLMs in the architecture. Experimental results demonstrate that our framework achieves a 7.3\% average reduction in cost compared to traditional Q-learning, with fewer training steps required for convergence.