Mem-T: Densifying Rewards for Long-Horizon Memory Agents

TL;DR

Mem-T introduces a hierarchical memory management system with a novel reinforcement learning framework, MoT-GRPO, enabling autonomous, efficient, and effective long-horizon memory operations for agents dealing with streaming inputs.

Contribution

The paper presents Mem-T, a new autonomous memory agent with dynamic memory updates and retrieval, and MoT-GRPO, a reinforcement learning method for dense training signals in long-horizon memory tasks.

Findings

Mem-T outperforms existing frameworks like A-Mem and Mem0 by up to 14.92%.

Mem-T reduces inference tokens per query by approximately 24.45% compared to GAM.

Mem-T achieves a favorable accuracy-efficiency trade-off.

Abstract

Memory agents, which depart from predefined memory-processing pipelines by endogenously managing the processing, storage, and retrieval of memories, have garnered increasing attention for their autonomy and adaptability. However, existing training paradigms remain constrained: agents often traverse long-horizon sequences of memory operations before receiving sparse and delayed rewards, which hinders truly end-to-end optimization of memory management policies. To address this limitation, we introduce Mem-T, an autonomous memory agent that interfaces with a lightweight hierarchical memory database to perform dynamic updates and multi-turn retrieval over streaming inputs. To effectively train long-horizon memory management capabilities, we further propose MoT-GRPO, a tree-guided reinforcement learning framework that transforms sparse terminal feedback into dense, step-wise supervision via memory operation tree backpropagation and hindsight credit assignment, thereby enabling the joint optimization of memory construction and retrieval. Extensive experiments demonstrate that Mem-T is (1) high-performing, surpassing frameworks such as A-Mem and Mem0 by up to $14.92\%$, and (2) economical, operating on a favorable accuracy-efficiency Pareto frontier and reducing inference tokens per query by $\sim24.45\%$ relative to GAM without sacrificing performance.