OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory

TL;DR

OCR-Memory introduces a visual-based memory system for long-horizon agents, enabling efficient retrieval of extended experiences with minimal prompt overhead by using optical encoding and visual anchors.

Contribution

It proposes a novel optical modality-based memory framework that enhances long-term experience retention and retrieval efficiency for autonomous agents.

Findings

Optical encoding increases effective memory capacity.

Retrieval via visual anchors reduces hallucination and preserves evidence.

Consistent performance gains on long-horizon benchmarks.

Abstract

Autonomous LLM agents increasingly operate in long-horizon, interactive settings where success depends on reusing experience accumulated over extended histories. However, existing agent memory systems are fundamentally constrained by text-context budgets: storing or revisiting raw trajectories is prohibitively token-expensive, while summarization and text-only retrieval trade token savings for information loss and fragmented evidence. To address this limitation, we propose Optical Context Retrieval Memory (OCR-Memory), a memory framework that leverages the visual modality as a high-density representation of agent experience, enabling retention of arbitrarily long histories with minimal prompt overhead at retrieval time. Specifically, OCR-Memory renders historical trajectories into images annotated with unique visual identifiers. OCR-Memory retrieves stored experience via a \emph{locate-and-transcribe} paradigm that selects relevant regions through visual anchors and retrieves the corresponding verbatim text, avoiding free-form generation and reducing hallucination. Experiments on long-horizon agent benchmarks show consistent gains under strict context limits, demonstrating that optical encoding increases effective memory capacity while preserving faithful evidence recovery.