A2DEPT: Large Language Model-Driven Automated Algorithm Design via Evolutionary Program Trees

TL;DR

This paper introduces A2DEPT, a novel system that uses evolutionary program trees and LLMs to autonomously design complete algorithms for combinatorial optimization, surpassing existing fixed-template methods.

Contribution

A2DEPT enables open-ended, system-level algorithm synthesis with evolutionary search and feedback repair, expanding beyond fixed-template heuristic design.

Findings

A2DEPT reduces the mean normalized optimality gap by 9.8% on standard benchmarks.

It outperforms existing LLM-based heuristic design methods.

The approach enables iterative refinement of complete algorithms.

Abstract

Designing heuristics for combinatorial optimization problems (COPs) is a fundamental yet challenging task that traditionally requires extensive domain expertise. Recently, Large Language Model (LLM)-based Automated Heuristic Design (AHD) has shown promise in autonomously generating heuristic components with minimal human intervention. However, most existing LLM-based AHD methods enforce fixed algorithmic templates to ensure executability, which confines the search to component-level tuning and limits system-level algorithmic expressiveness. To enable open-ended solver synthesis beyond rigid templates, we propose Automated Algorithm Design via Evolutionary Program Trees (A2DEPT), which treats LLMs as system-level algorithm architects. A2DEPT explores the vast program space via a tree-structured evolutionary search with hybrid selection and hierarchical operators, enabling iterative refinement of complete algorithms. To make open-ended generation practical, we enforce executability with a lightweight program-maintenance loop that performs feedback-driven repair. In experiments, A2DEPT consistently outperforms representative LLM-based baselines on both standard and highly constrained benchmarks. On the standard benchmarks, it reduces the mean normalized optimality gap by 9.8% relative to the strongest competing AHD baseline.