A Survey of Automatic Prompt Optimization with Instruction-focused Heuristic-based Search Algorithm
Wendi Cui, Zhuohang Li, Hao Sun, Damien Lopez, Kamalika Das, Bradley A. Malin, Sricharan Kumar, Jiaxin Zhang

TL;DR
This survey reviews heuristic-based search algorithms for automatic prompt optimization in large language models, categorizing methods, datasets, tools, and discussing future challenges for enhancing prompt refinement.
Contribution
It provides a comprehensive taxonomy of automatic prompt optimization methods, categorizing them by optimization points, criteria, operators, and algorithms, and highlights supporting datasets and tools.
Findings
Categorizes prompt optimization methods systematically
Identifies key datasets and tools for prompt refinement
Discusses open challenges and future directions
Abstract
Recent advances in Large Language Models have led to remarkable achievements across a variety of Natural Language Processing tasks, making prompt engineering increasingly central to guiding model outputs. While manual methods can be effective, they typically rely on intuition and do not automatically refine prompts over time. In contrast, automatic prompt optimization employing heuristic-based search algorithms can systematically explore and improve prompts with minimal human oversight. This survey proposes a comprehensive taxonomy of these methods, categorizing them by where optimization occurs, what is optimized, what criteria drive the optimization, which operators generate new prompts, and which iterative search algorithms are applied. We further highlight specialized datasets and tools that support and accelerate automated prompt refinement. We conclude by discussing key open…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEducational Technology and Assessment · Teaching and Learning Programming · Intelligent Tutoring Systems and Adaptive Learning
