Generalizable Self-Evolving Memory for Automatic Prompt Optimization

TL;DR

MemAPO introduces a memory-driven, self-evolving framework for prompt optimization that enhances generalization, reuses strategies, and reduces costs by learning from past reasoning trajectories and errors in large language models.

Contribution

It presents a novel dual-memory mechanism enabling prompt optimization to be generalizable and self-improving through experience accumulation, unlike traditional task-specific methods.

Findings

Outperforms existing prompt optimization methods on various benchmarks.

Reduces optimization costs significantly.

Enables continuous improvement through self-reflection and memory editing.

Abstract

Automatic prompt optimization is a promising approach for adapting large language models (LLMs) to downstream tasks, yet existing methods typically search for a specific prompt specialized to a fixed task. This paradigm limits generalization across heterogeneous queries and prevents models from accumulating reusable prompting knowledge over time. In this paper, we propose MemAPO, a memory-driven framework that reconceptualizes prompt optimization as generalizable and self-evolving experience accumulation. MemAPO maintains a dual-memory mechanism that distills successful reasoning trajectories into reusable strategy templates while organizing incorrect generations into structured error patterns that capture recurrent failure modes. Given a new prompt, the framework retrieves both relevant strategies and failure patterns to compose prompts that promote effective reasoning while discouraging known mistakes. Through iterative self-reflection and memory editing, MemAPO continuously updates its memory, enabling prompt optimization to improve over time rather than restarting from scratch for each task. Experiments on diverse benchmarks show that MemAPO consistently outperforms representative prompt optimization baselines while substantially reducing optimization cost.