Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management

TL;DR

This paper introduces a decision-theoretic framework for managing external memory in large language models, emphasizing long-term utility and uncertainty, moving beyond traditional heuristics to enable more principled memory decisions.

Contribution

It proposes DAM, a framework that models memory management as a sequential decision process, providing a foundation for future uncertainty-aware memory systems.

Findings

Reframes memory management as a decision-theoretic problem

Highlights limitations of heuristic approaches

Provides a foundation for future research on uncertainty-aware memory

Abstract

External memory is a key component of modern large language model (LLM) systems, enabling long-term interaction and personalization. Despite its importance, memory management is still largely driven by hand-designed heuristics, offering little insight into the long-term and uncertain consequences of memory decisions. In practice, choices about what to read or write shape future retrieval and downstream behavior in ways that are difficult to anticipate. We argue that memory management should be viewed as a sequential decision-making problem under uncertainty, where the utility of memory is delayed and dependent on future interactions. To this end, we propose DAM (Decision-theoretic Agent Memory), a decision-theoretic framework that decomposes memory management into immediate information access and hierarchical storage maintenance. Within this architecture, candidate operations are evaluated via value functions and uncertainty estimators, enabling an aggregate policy to arbitrate decisions based on estimated long-term utility and risk. Our contribution is not a new algorithm, but a principled reframing that clarifies the limitations of heuristic approaches and provides a foundation for future research on uncertainty-aware memory systems.