CDEoH: Category-Driven Automatic Algorithm Design With Large Language Models

TL;DR

This paper introduces CDEoH, a novel approach leveraging large language models to explicitly incorporate algorithm category diversity, improving stability and performance in automated algorithm design for combinatorial optimization.

Contribution

CDEoH explicitly models algorithm categories and balances diversity and performance, enabling parallel exploration and enhancing evolutionary stability in LLM-based heuristic search.

Findings

CDEoH mitigates convergence to a single evolutionary path.

It significantly improves stability and average performance.

Effective across multiple problem scales and types.

Abstract

With the rapid advancement of large language models (LLMs), LLM-based heuristic search methods have demonstrated strong capabilities in automated algorithm generation. However, their evolutionary processes often suffer from instability and premature convergence. Existing approaches mainly address this issue through prompt engineering or by jointly evolving thought and code, while largely overlooking the critical role of algorithmic category diversity in maintaining evolutionary stability. To this end, we propose Category Driven Automatic Algorithm Design with Large Language Models (CDEoH), which explicitly models algorithm categories and jointly balances performance and category diversity in population management, enabling parallel exploration across multiple algorithmic paradigms. Extensive experiments on representative combinatorial optimization problems across multiple scales demonstrate that CDEoH effectively mitigates convergence toward a single evolutionary direction, significantly enhancing evolutionary stability and achieving consistently superior average performance across tasks and scales.