Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory

TL;DR

TriMem introduces a multi-granularity memory system for LLM agents, enhancing long-term dialogue retention and reasoning by combining raw segments, atomic facts, and holistic profiles, optimized through response feedback.

Contribution

The paper proposes TriMem, a novel memory architecture with three coexisting representations and a prompt optimization method, improving over fact-centric approaches for lifelong LLM interaction.

Findings

TriMem outperforms strong baselines on LoCoMo and PerLTQA datasets.

The approach enables lifelong evolution without parameter updates.

Extensive experiments validate the effectiveness across multiple LLM backbones.

Abstract

To enable reliable long-term interaction, LLM agents require a memory system that can faithfully store, efficiently retrieve, and deeply reason over accumulated dialogue history. Most existing methods adopt an extracted fact based paradigm: handcrafted static prompts compress raw dialogues into atomic facts, which are then stored, matched, and injected into downstream reasoning. Nevertheless, such fact-centric designs inevitably discard fine-grained details in original dialogues and fail to support deep reasoning over scattered isolated facts. Moreover, static prompts cannot maintain consistent extraction granularity across diverse dialogue styles. To address these limitations, we propose TriMem, which maintains three coexisting representation granularities, including raw dialogue segments anchored by source identifiers for storage fidelity, extracted atomic facts for efficient memory retrieval, synthesized profiles that aggregate dispersed facts into holistic semantic understanding for deep reasoning. We further adopt TextGrad-based prompt optimization, which iteratively refines extraction and profiling prompts via response quality feedback, achieving lifelong evolution without any parameter updating. Extensive experiments on LoCoMo and PerLTQA across multiple LLM backbones demonstrate that TriMem consistently outperforms strong memory baselines. The code is available at https://TMLR-TriMem.github.io .