Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory

TL;DR

This paper provides a unified framework and empirical analysis of long-term dialog memory architectures, revealing that many performance differences stem from system settings rather than architectural innovations.

Contribution

It introduces a comprehensive framework for dialog memory systems and systematically evaluates the impact of design choices, establishing reliable baselines for future research.

Findings

Performance differences often due to system settings

Graph-based approaches are not always superior

Stable baselines identified for future work

Abstract

Graph structures are increasingly used in dialog memory systems, but empirical findings on their effectiveness remain inconsistent, making it unclear which design choices truly matter. We present an experimental, system-oriented analysis of long-term dialog memory architectures. We introduce a unified framework that decomposes dialog memory systems into core components and supports both graph-based and non-graph approaches. Under this framework, we conduct controlled, stage-wise experiments on LongMemEval and HaluMem, comparing common design choices in memory representation, organization, maintenance, and retrieval. Our results show that many performance differences are driven by foundational system settings rather than specific architectural innovations. Based on these findings, we identify stable and reliable strong baselines for future dialog memory research. Code are available at https://github.com/AvatarMemory/UnifiedMem