HiFo-Prompt: Prompting with Hindsight and Foresight for LLM-based Automatic Heuristic Design

TL;DR

HiFo-Prompt introduces a dual prompting framework for LLM-based automatic heuristic design that adaptively guides search and distills successful heuristics, significantly improving performance and efficiency in evolutionary computation tasks.

Contribution

The paper presents HiFo-Prompt, a novel framework combining foresight and hindsight prompts to enhance LLM-guided heuristic design in evolutionary algorithms.

Findings

Outperforms state-of-the-art LLM-based AHD methods

Generates higher-quality heuristics

Achieves faster convergence and better query efficiency

Abstract

LLM-based Automatic Heuristic Design (AHD) within Evolutionary Computation (EC) frameworks has shown promising results. However, its effectiveness is hindered by the use of static operators and the lack of knowledge accumulation mechanisms. We introduce HiFo-Prompt, a framework that guides LLMs with two synergistic prompting strategies: Foresight and Hindsight. Foresight-based prompts adaptively steer the search based on population dynamics, managing the exploration-exploitation trade-off. In addition, hindsight-based prompts mimic human expertise by distilling successful heuristics from past generations into fundamental, reusable design principles. This dual mechanism transforms transient discoveries into a persistent knowledge base, enabling the LLM to learn from its own experience. Empirical results demonstrate that HiFo-Prompt significantly outperforms state-of-the-art LLM-based AHD methods, generating higher-quality heuristics while achieving substantially faster convergence and superior query efficiency. Our code is available at https://github.com/Challenger-XJTU/HiFo-Prompt.