Goal-Directed Search Outperforms Goal-Agnostic Memory Compression in Long-Context Memory Tasks

TL;DR

This paper introduces SUMER, a reinforcement learning agent that uses goal-directed search to access uncompressed memory, outperforming traditional compression methods in long-context memory tasks.

Contribution

The paper presents SUMER, a novel goal-directed search approach that surpasses existing memory compression techniques in long-context understanding tasks.

Findings

SUMER outperforms all biased memory compression approaches.

SUMER achieves state-of-the-art performance with 43% gain.

Goal-directed search on raw data surpasses goal-agnostic compression methods.

Abstract

How to enable human-like long-term memory in large language models (LLMs) has been a central question for unlocking more general capabilities such as few-shot generalization. Existing memory frameworks and benchmarks focus on finding the optimal memory compression algorithm for higher performance in tasks that require recollection and sometimes further reasoning. However, such efforts have ended up building more human bias into the compression algorithm, through the search for the best prompts and memory architectures that suit specific benchmarks, rather than finding a general solution that would work on other data distributions. On the other hand, goal-directed search on uncompressed information could potentially exhibit superior performance because compression is lossy, and a predefined compression algorithm will not fit all raw data distributions. Here we present SUMER (Search in Uncompressed Memory via Experience Replay), an end-to-end reinforcement learning agent with verifiable reward (RLVR) that learns to use search tools to gather information and answer a target question. On the LoCoMo dataset for long-context conversation understanding, SUMER with Qwen2.5-7B-Instruct learned to use search tools and outperformed all other biased memory compression approaches and also the full-context baseline, reaching SOTA performance (43% gain over the prior best). We demonstrate that a simple search method applied to raw data outperforms goal-agnostic and biased compression algorithms in current long-context memory tasks, arguing for new paradigms and benchmarks that are more dynamic and autonomously scalable. Code for SUMER and all implemented baselines is publicly available at https://github.com/zycyc/SUMER.