Chain-of-Memory: Lightweight Memory Construction with Dynamic Evolution for LLM Agents

TL;DR

The paper introduces CoM, a lightweight, dynamic memory framework for LLM agents that improves reasoning accuracy and reduces computational costs by organizing retrieved data into coherent inference paths.

Contribution

Proposes CoM, a novel lightweight memory system with dynamic evolution and adaptive pruning, outperforming existing methods in accuracy and efficiency.

Findings

CoM achieves 7.5%-10.4% accuracy gains on benchmarks.

Reduces token consumption to 2.7% of complex architectures.

Decreases latency to 6.0% of traditional memory systems.

Abstract

External memory systems are pivotal for enabling Large Language Model (LLM) agents to maintain persistent knowledge and perform long-horizon decision-making. Existing paradigms typically follow a two-stage process: computationally expensive memory construction (e.g., structuring data into graphs) followed by naive retrieval-augmented generation. However, our empirical analysis reveals two fundamental limitations: complex construction incurs high costs with marginal performance gains, and simple context concatenation fails to bridge the gap between retrieval recall and reasoning accuracy. To address these challenges, we propose CoM (Chain-of-Memory), a novel framework that advocates for a paradigm shift toward lightweight construction paired with sophisticated utilization. CoM introduces a Chain-of-Memory mechanism that organizes retrieved fragments into coherent inference paths through dynamic evolution, utilizing adaptive truncation to prune irrelevant noise. Extensive experiments on the LongMemEval and LoCoMo benchmarks demonstrate that CoM outperforms strong baselines with accuracy gains of 7.5%-10.4%, while drastically reducing computational overhead to approximately 2.7% of token consumption and 6.0% of latency compared to complex memory architectures.