Autonomous Prompt Engineering in Large Language Models

TL;DR

This paper presents APET, a toolbox enabling GPT-4 to autonomously optimize prompts using advanced strategies, significantly improving performance on various tasks without external data, marking a major advancement in autonomous AI systems.

Contribution

Introduction of APET, a novel framework allowing GPT-4 to autonomously perform prompt engineering, enhancing task performance through sophisticated strategies without external data.

Findings

Improved task performance in Word Sorting (+4.4%) and Geometric Shapes (+6.8%)

Demonstrated autonomous prompt optimization without external data

Identified challenges in complex tasks like Checkmate in One (-14.8%)

Abstract

Prompt engineering is a crucial yet challenging task for optimizing the performance of large language models (LLMs) on customized tasks. This pioneering research introduces the Automatic Prompt Engineering Toolbox (APET), which enables GPT-4 to autonomously apply prompt engineering techniques. By leveraging sophisticated strategies such as Expert Prompting, Chain of Thought, and Tree of Thoughts, APET empowers GPT-4 to dynamically optimize prompts, resulting in substantial improvements in tasks like Word Sorting (4.4% increase) and Geometric Shapes (6.8% increase). Despite encountering challenges in complex tasks such as Checkmate in One (-14.8%), these findings demonstrate the transformative potential of APET in automating complex prompt optimization processes without the use of external data. Overall, this research represents a significant leap in AI development, presenting a robust framework for future innovations in autonomous AI systems and highlighting the ability of GPT-4 to bring prompt engineering theory to practice. It establishes a foundation for enhancing performance in complex task performance and broadening the practical applications of these techniques in real-world scenarios.