RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents

TL;DR

RecMem introduces a recurrence-based memory consolidation method for long-running LLM agents, significantly reducing token costs while maintaining or improving memory accuracy by selectively invoking LLMs based on interaction recurrence.

Contribution

It proposes a novel recurrence-based memory consolidation approach that minimizes token consumption and enhances memory accuracy in long-running LLM agents.

Findings

RecMem reduces memory construction token cost by up to 87%.

RecMem exceeds the accuracy of three state-of-the-art memory systems.

Selective invocation of LLMs based on recurrence improves efficiency.

Abstract

Memory systems often organize user-agent interactions as retrievable external memory and are crucial for long-running agents by overcoming the limited context windows of LLMs. However, existing memory systems invoke LLMs to process every incoming interaction for memory extraction, and such an eager memory consolidation scheme leads to substantial token consumption. To tackle this problem, we propose RecMem by rethinking when memory consolidation should be conducted. RecMem stores incoming interactions in a subconscious memory layer and encode them using lightweight embedding models for retrieval. LLMs are only invoked to extract episodic and semantic memory when sustained recurrence are observed for semantically similar interactions. Such recurrence-based consolidation works because these interactions correspond to a semantic cluster with rich information and thus are worth extraction and summarization. To improve accuracy, RecMem also incorporates a semantic refinement mechanism that recovers the fine-grained facts omitted by memory extraction. Experiments show that RecMem reduces the memory construction token cost of three SOTA memory systems by up to 87% while exceeding their accuracy.