EoH-S: Evolution of Heuristic Set using LLMs for Automated Heuristic Design

TL;DR

This paper introduces EoH-S, a method that uses large language models to evolve a set of complementary heuristics for automated heuristic design, significantly improving performance across diverse problem instances.

Contribution

It proposes a novel formulation for generating heuristic sets with LLMs and introduces EoH-S, which effectively evolves high-quality, complementary heuristics for diverse problem instances.

Findings

EoH-S outperforms existing methods on three AHD tasks.

Achieves up to 60% performance improvements.

Demonstrates effectiveness across various sizes and distributions.

Abstract

Automated Heuristic Design (AHD) using Large Language Models (LLMs) has achieved notable success in recent years. Despite the effectiveness of existing approaches, they only design a single heuristic to serve all problem instances, often inducing poor generalization across different distributions or settings. To address this issue, we propose Automated Heuristic Set Design (AHSD), a new formulation for LLM-driven AHD. The aim of AHSD is to automatically generate a small-sized complementary heuristic set to serve diverse problem instances, such that each problem instance could be optimized by at least one heuristic in this set. We show that the objective function of AHSD is monotone and supermodular. Then, we propose Evolution of Heuristic Set (EoH-S) to apply the AHSD formulation for LLM-driven AHD. With two novel mechanisms of complementary population management and complementary-aware memetic search, EoH-S could effectively generate a set of high-quality and complementary heuristics. Comprehensive experimental results on three AHD tasks with diverse instances spanning various sizes and distributions demonstrate that EoH-S consistently outperforms existing state-of-the-art AHD methods and achieves up to 60\% performance improvements.