A Systematic Survey of Automatic Prompt Optimization Techniques

TL;DR

This paper provides a comprehensive survey of Automatic Prompt Optimization techniques for large language models, categorizing existing methods and highlighting challenges to guide future research in prompt engineering automation.

Contribution

It introduces a formal definition of APO, presents a unifying framework, and systematically categorizes existing works to advance understanding and development in this field.

Findings

APO techniques improve LLM performance across tasks.

The survey identifies key challenges and research gaps.

A unifying framework for APO methods is proposed.

Abstract

Since the advent of large language models (LLMs), prompt engineering has been a crucial step for eliciting desired responses for various Natural Language Processing (NLP) tasks. However, prompt engineering remains an impediment for end users due to rapid advances in models, tasks, and associated best practices. To mitigate this, Automatic Prompt Optimization (APO) techniques have recently emerged that use various automated techniques to help improve the performance of LLMs on various tasks. In this paper, we present a comprehensive survey summarizing the current progress and remaining challenges in this field. We provide a formal definition of APO, a 5-part unifying framework, and then proceed to rigorously categorize all relevant works based on their salient features therein. We hope to spur further research guided by our framework.