Memp: Exploring Agent Procedural Memory

TL;DR

This paper introduces Memp, a learnable and updatable procedural memory system for LLM-based agents, improving their task success and efficiency through continuous memory refinement.

Contribution

Memp enables agents to develop a dynamic, lifelong procedural memory that enhances performance and can transfer across models, a novel approach in agent memory management.

Findings

Agents with Memp show higher success rates on TravelPlanner and ALFWorld.

Procedural memory refinement leads to increased efficiency in task execution.

Memory transfer from stronger to weaker models retains performance gains.

Abstract

Large Language Models (LLMs) based agents excel at diverse tasks, yet they suffer from brittle procedural memory that is manually engineered or entangled in static parameters. In this work, we investigate strategies to endow agents with a learnable, updatable, and lifelong procedural memory. We propose Memp that distills past agent trajectories into both fine-grained, step-by-step instructions and higher-level, script-like abstractions, and explore the impact of different strategies for Build, Retrieval, and Update of procedural memory. Coupled with a dynamic regimen that continuously updates, corrects, and deprecates its contents, this repository evolves in lockstep with new experience. Empirical evaluation on TravelPlanner and ALFWorld shows that as the memory repository is refined, agents achieve steadily higher success rates and greater efficiency on analogous tasks. Moreover, procedural memory built from a stronger model retains its value: migrating the procedural memory to a weaker model can also yield substantial performance gains. Code is available at https://github.com/zjunlp/MemP.