Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents

TL;DR

This paper introduces Temporal Semantic Memory (TSM), a novel memory framework for personalized LLM agents that models semantic time and durative memory to improve temporal accuracy and consistency in dialogue understanding.

Contribution

It proposes a new memory framework that constructs semantic timelines and durative memories, addressing temporal inaccuracy and fragmentation in existing methods.

Findings

TSM outperforms existing methods on LongMemEval and LoCoMo datasets.

Achieves up to 12.2% accuracy improvement.

Effectively models temporal and durative aspects of memory.

Abstract

Memory enables Large Language Model (LLM) agents to perceive, store, and use information from past dialogues, which is essential for personalization. However, existing methods fail to properly model the temporal dimension of memory in two aspects: 1) Temporal inaccuracy: memories are organized by dialogue time rather than their actual occurrence time; 2) Temporal fragmentation: existing methods focus on point-wise memory, losing durative information that captures persistent states and evolving patterns. To address these limitations, we propose Temporal Semantic Memory (TSM), a memory framework that models semantic time for point-wise memory and supports the construction and utilization of durative memory. During memory construction, it first builds a semantic timeline rather than a dialogue one. Then, it consolidates temporally continuous and semantically related information into a durative memory. During memory utilization, it incorporates the query's temporal intent on the semantic timeline, enabling the retrieval of temporally appropriate durative memories and providing time-valid, duration-consistent context to support response generation. Experiments on LongMemEval and LoCoMo show that TSM consistently outperforms existing methods and achieves up to 12.2% absolute improvement in accuracy, demonstrating the effectiveness of the proposed method.