R^2-Mem: Reflective Experience for Memory Search

TL;DR

R^2-Mem introduces a reflective experience framework for memory search that improves accuracy and efficiency of agents by learning from past search trajectories without reinforcement learning.

Contribution

The paper presents R^2-Mem, a novel RL-free, low-cost framework that enhances memory search systems through reflective experience and self-guided improvement.

Findings

F1 scores improved by up to 22.6%

Token consumption reduced by 12.9%

Search iterations decreased by 20.2%

Abstract

Deep search has recently emerged as a promising paradigm for enabling agents to retrieve fine-grained historical information without heavy memory pre-managed. However, existing deep search agents for memory system repeat past error behaviors because they fail to learn from the prior high- and low-quality search trajectories. To address this limitation, we propose R^2-Mem, a reflective experience framework for memory search systems. In the offline stage, a Rubric-guided Evaluator scores low- and high-quality steps in historical trajectories, and a self-Reflection Learner distills the corresponding abstract experience. During the online inference, the retrieved experience will guide future search actions to avoid repeated mistakes and maintain high-quality behaviors. Extensive experiments demonstrate that R^2-Mem consistently improves both effectiveness and efficiency over strong baselines, improving F1 scores by up to 22.6%, while reducing token consumption by 12.9% and search iterations by 20.2%. These results verify that R^2-Mem provides a RL-free and low-cost solution for self-improving LLM agents.