Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimization

TL;DR

MAPO introduces a momentum-based prompt optimization method for LLMs that improves convergence speed and accuracy by tracking gradient history and using advanced search algorithms.

Contribution

The paper presents MAPO, a novel prompt optimization technique that enhances efficiency and effectiveness over previous methods like ProTeGi through momentum and strategic search.

Findings

Faster convergence with fewer API calls

Higher F1 scores compared to ProTeGi

Robust and scalable prompt engineering solution

Abstract

Momentum-Aided Prompt Optimization (MAPO) enhances the efficiency and efficacy of prompt optimization for Large Language Models (LLMs). Building on ProTeGi, MAPO uses positive natural language "gradients" and a momentum-based extension to refine prompts effectively. By tracking gradient history, MAPO avoids local minima and oscillations. It also utilizes beam search and an Upper Confidence Bound (UCB) algorithm for balanced candidate expansion and selection. Benchmark testing shows that MAPO achieves faster convergence time with fewer API calls and higher F1 scores than ProTeGi, proving it as a robust and scalable solution for automated prompt engineering in LLMs.