Neuromem: A Granular Decomposition of the Streaming Lifecycle in External Memory for LLMs
Ruicheng Zhang, Xinyi Li, Tianyi Xu, Shuhao Zhang, Xiaofei Liao, Hai Jin

TL;DR
Neuromem introduces a comprehensive benchmarking framework for streaming external memory modules in large language models, analyzing their lifecycle and performance under dynamic, real-time data ingestion and retrieval scenarios.
Contribution
It provides a scalable testbed that decomposes the memory lifecycle into five dimensions and evaluates various memory strategies on multiple datasets.
Findings
Memory performance degrades as memory size increases.
Time-related queries are the most challenging.
Memory data structure significantly impacts quality and efficiency.
Abstract
Most evaluations of External Memory Module assume a static setting: memory is built offline and queried at a fixed state. In practice, memory is streaming: new facts arrive continuously, insertions interleave with retrievals, and the memory state evolves while the model is serving queries. In this regime, accuracy and cost are governed by the full memory lifecycle, which encompasses the ingestion, maintenance, retrieval, and integration of information into generation. We present Neuromem, a scalable testbed that benchmarks External Memory Modules under an interleaved insertion-and-retrieval protocol and decomposes its lifecycle into five dimensions including memory data structure, normalization strategy, consolidation policy, query formulation strategy, and context integration mechanism. Using three representative datasets LOCOMO, LONGMEMEVAL, and MEMORYAGENTBENCH, Neuromem evaluates…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInformation Retrieval and Search Behavior · Advanced Database Systems and Queries · Semantic Web and Ontologies
