RedAHD: Reduction-Based End-to-End Automatic Heuristic Design with Large Language Models

TL;DR

RedAHD introduces an end-to-end framework using large language models to automate heuristic design for NP-hard combinatorial optimization problems, reducing manual effort and improving solution quality.

Contribution

It presents a novel reduction-based, end-to-end approach that enables LLMs to directly design heuristics without predefined frameworks, streamlining the process.

Findings

RedAHD achieves competitive or better results than state-of-the-art methods.

The framework reduces manual intervention in heuristic design.

Effective across six different combinatorial optimization problems.

Abstract

Solving NP-hard combinatorial optimization problems (COPs) (e.g., traveling salesman problems (TSPs) and capacitated vehicle routing problems (CVRPs)) in practice traditionally involves handcrafting heuristics or specifying a search space for finding effective heuristics. The main challenges from these approaches, however, are the sheer amount of domain knowledge and implementation efforts required from human experts. Recently, significant progress has been made to address these challenges, particularly by using large language models (LLMs) to design heuristics within some predetermined generalized algorithmic framework (GAF, e.g., ant colony optimization and guided local search) for building key functions/components (e.g., a priori information on how promising it is to include each edge in a solution for TSP and CVRP). Although existing methods leveraging this idea have shown to yield impressive optimization performance, they are not fully end-to-end and still require considerable manual interventions. In this paper, we propose a novel end-to-end framework, named RedAHD, that enables these LLM-based heuristic design methods to operate without the need of GAFs. More specifically, RedAHD employs LLMs to automate the process of reduction, i.e., transforming the COP at hand into similar COPs that are better-understood, from which LLM-based heuristic design methods can design effective heuristics for directly solving the transformed COPs and, in turn, indirectly solving the original COP. Our experimental results, evaluated on six COPs, show that RedAHD is capable of designing heuristics with competitive or improved results over the state-of-the-art methods with minimal human involvement.