Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces

TL;DR

This paper introduces the Generative Semantic Workspace, a neuro-inspired memory framework that enhances large language models' ability to reason over long, evolving narratives by building structured, interpretable episodic representations.

Contribution

The paper presents GSW, a novel generative memory framework that constructs structured, interpretable episodic representations, enabling LLMs to track entities and events over extended contexts.

Findings

GSW outperforms existing RAG baselines by up to 20% on EpBench.

GSW reduces query-time tokens by 51%, improving efficiency.

GSW demonstrates human-like episodic memory capabilities in LLMs.

Abstract

Large Language Models (LLMs) face fundamental challenges in long-context reasoning: many documents exceed their finite context windows, while performance on texts that do fit degrades with sequence length, necessitating their augmentation with external memory frameworks. Current solutions, which have evolved from retrieval using semantic embeddings to more sophisticated structured knowledge graphs representations for improved sense-making and associativity, are tailored for fact-based retrieval and fail to build the space-time-anchored narrative representations required for tracking entities through episodic events. To bridge this gap, we propose the \textbf{Generative Semantic Workspace} (GSW), a neuro-inspired generative memory framework that builds structured, interpretable representations of evolving situations, enabling LLMs to reason over evolving roles, actions, and spatiotemporal contexts. Our framework comprises an \textit{Operator}, which maps incoming observations to intermediate semantic structures, and a \textit{Reconciler}, which integrates these into a persistent workspace that enforces temporal, spatial, and logical coherence. On the Episodic Memory Benchmark (EpBench) \cite{huet_episodic_2025} comprising corpora ranging from 100k to 1M tokens in length, GSW outperforms existing RAG based baselines by up to \textbf{20\%}. Furthermore, GSW is highly efficient, reducing query-time context tokens by \textbf{51\%} compared to the next most token-efficient baseline, reducing inference time costs considerably. More broadly, GSW offers a concrete blueprint for endowing LLMs with human-like episodic memory, paving the way for more capable agents that can reason over long horizons. Code is available at https://github.com/roychowdhuryresearch/gsw-memory.