Mem-$\pi$: Adaptive Memory through Learning When and What to Generate

TL;DR

Mem-$$ introduces an adaptive memory framework for large language model agents that generates context-specific guidance on demand, improving task performance over traditional retrieval methods.

Contribution

It proposes a novel decision-guided generation approach with reinforcement learning, enabling dynamic, context-aware guidance in complex tasks.

Findings

Achieves over 30% relative improvement on web navigation tasks.

Outperforms retrieval-based and prior RL-based memory baselines.

Demonstrates effectiveness across diverse agentic benchmarks.

Abstract

We present Mem-$\pi$, a framework for adaptive memory in large language model (LLM) agents, where useful guidance is generated on demand rather than retrieved from external memory stores. Existing memory-augmented agents typically rely on similarity-based retrieval from episodic memory banks or skill libraries, returning static entries that often misalign with the current context. In contrast, Mem-$\pi$ uses a dedicated language or vision-language model with its own parameters, separate from the downstream agent, to generate context-specific guidance for complex tasks. Conditioned on the current agent context, the model jointly decides when to produce guidance and what guidance to produce. We train it with a decision-content decoupled reinforcement learning (RL) objective, enabling it to abstain when generation would not help and otherwise produce concise, useful guidance. Across diverse agentic benchmarks spanning web navigation, terminal-based tool use, and text-based embodied interaction, Mem-$\pi$ consistently outperforms retrieval-based and prior RL-optimized memory baselines, achieving over 30% relative improvement on web navigation tasks.