Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory

TL;DR

This paper introduces a decision-centric rate-distortion framework for agent memory, emphasizing the preservation of decision-critical distinctions over descriptive accuracy, and proposes DeMem, an online memory learner with theoretical guarantees.

Contribution

It formulates a novel decision-focused memory optimization framework and develops DeMem, an online algorithm with provable near-minimax regret guarantees for long-horizon agents.

Findings

DeMem achieves consistent decision quality improvements under fixed memory budgets.

The framework provides an exact forgetting boundary and a memory-distortion tradeoff characterization.

DeMem outperforms baseline methods on synthetic and conversational benchmarks.

Abstract

Long-horizon language agents must operate under limited runtime memory, yet existing memory mechanisms often organize experience around descriptive criteria such as relevance, salience, or summary quality. For an agent, however, memory is valuable not because it faithfully describes the past, but because it preserves the distinctions between histories that must remain separated under a fixed budget to support good decisions. We cast this as a decision-centric rate-distortion problem, measuring memory quality by the loss in achievable decision quality induced by compression. This yields an exact forgetting boundary for what can be safely forgotten, and a memory-distortion frontier characterizing the optimal tradeoff between memory budget and decision quality. Motivated by this decision-centric view of memory, we propose DeMem, an online memory learner that refines its partition only when data certify that a shared state would induce decision conflict, and prove near-minimax regret guarantees. On both controlled synthetic diagnostics and long-horizon conversational benchmarks, DeMem yields consistent gains under the same runtime budget, supporting the principle that memory should preserve the distinctions that matter for decisions, not descriptions.