Grounding Agent Memory in Contextual Intent

TL;DR

This paper introduces STITCH, a memory system for large language models that uses structured intent cues to improve context-aware retrieval in long, goal-oriented interactions, significantly reducing retrieval errors.

Contribution

The paper presents STITCH, a novel intent-indexed memory system that enhances long-horizon reasoning by filtering relevant history based on structured contextual intent cues.

Findings

STITCH outperforms baselines by 35.6% on CAME-Bench.

Intent indexing reduces retrieval noise in long trajectories.

STITCH achieves state-of-the-art results on multiple benchmarks.

Abstract

Deploying large language models in long-horizon, goal-oriented interactions remains challenging because similar entities and facts recur under different latent goals and constraints, causing memory systems to retrieve context-mismatched evidence. We propose STITCH (Structured Intent Tracking in Contextual History), an agentic memory system that indexes each trajectory step with a structured retrieval cue, contextual intent, and retrieves history by matching the current step's intent. Contextual intent provides compact signals that disambiguate repeated mentions and reduce interference: (1) the current latent goal defining a thematic segment, (2) the action type, and (3) the salient entity types anchoring which attributes matter. During inference, STITCH filters and prioritizes memory snippets by intent compatibility, suppressing semantically similar but context-incompatible history. For evaluation, we introduce CAME-Bench, a benchmark for context-aware retrieval in realistic, dynamic, goal-oriented trajectories. Across CAME-Bench and LongMemEval, STITCH achieves state-of-the-art performance, outperforming the strongest baseline by 35.6%, with the largest gains as trajectory length increases. Our analysis shows that intent indexing substantially reduces retrieval noise, supporting intent-aware memory for robust long-horizon reasoning.