Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

TL;DR

Evo-Memory introduces a new benchmark and framework for evaluating how large language model agents can dynamically learn, adapt, and improve their memory during continuous task streams, addressing a key gap in current static evaluation methods.

Contribution

The paper presents Evo-Memory, a comprehensive streaming benchmark for testing self-evolving memory in LLM agents, along with implementations of memory modules and a new method ReMem for continual learning.

Findings

Evaluated over ten memory modules across ten diverse datasets.

Demonstrated the effectiveness of ReMem in continual improvement.

Provided insights into memory management for long-term LLM agent deployment.

Abstract

Statefulness is essential for large language model (LLM) agents to perform long-term planning and problem-solving. This makes memory a critical component, yet its management and evolution remain largely underexplored. Existing evaluations mostly focus on static conversational settings, where memory is passively retrieved from dialogue to answer queries, overlooking the dynamic ability to accumulate and reuse experience across evolving task streams. In real-world environments such as interactive problem assistants or embodied agents, LLMs are required to handle continuous task streams, yet often fail to learn from accumulated interactions, losing valuable contextual insights, a limitation that calls for test-time evolution, where LLMs retrieve, integrate, and update memory continuously during deployment. To bridge this gap, we introduce Evo-Memory, a comprehensive streaming benchmark and framework for evaluating self-evolving memory in LLM agents. Evo-Memory structures datasets into sequential task streams, requiring LLMs to search, adapt, and evolve memory after each interaction. We unify and implement over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets. To better benchmark experience reuse, we provide a baseline method, ExpRAG, for retrieving and utilizing prior experience, and further propose ReMem, an action-think-memory refine pipeline that tightly integrates reasoning, task actions, and memory updates to achieve continual improvement.