Contextual Agentic Memory is a Memo, Not True Memory

TL;DR

Current AI agentic memory systems only perform lookup and do not truly implement memory, leading to limitations in learning, generalization, and security, as explained through neuroscience principles.

Contribution

The paper argues that existing AI memory systems are category errors, and proposes a neuro-inspired dual-system approach for genuine memory implementation.

Findings

Lookup-based systems face a generalization ceiling.

Weight-based memory allows for abstract rule application.

Current systems are vulnerable to memory poisoning.

Abstract

Current agentic memory systems (vector stores, retrieval-augmented generation, scratchpads, and context-window management) do not implement memory: they implement lookup. We argue that treating lookup as memory is a category error with provable consequences for agent capability, long-term learning, and security. Retrieval generalizes by similarity to stored cases; weight-based memory generalizes by applying abstract rules to inputs never seen before. Conflating the two produces agents that accumulate notes indefinitely without developing expertise, face a provable generalization ceiling on compositionally novel tasks that no increase in context size or retrieval quality can overcome, and are structurally vulnerable to persistent memory poisoning as injected content propagates across all future sessions. Drawing on Complementary Learning Systems theory from neuroscience, we show that biological intelligence solved this problem by pairing fast hippocampal exemplar storage with slow neocortical weight consolidation, and that current AI agents implement only the first half. We formalize these limitations, address four alternative views, and close with a co-existence proposal and a call to action for system builders, benchmark designers, and the memory community.