MemReranker: Reasoning-Aware Reranking for Agent Memory Retrieval

TL;DR

MemReranker is a reasoning-aware reranking model for agent memory retrieval that improves relevance calibration and reasoning capabilities, outperforming existing models on benchmark tasks with lower latency.

Contribution

Introduces MemReranker, a multi-stage distillation-based reranker with enhanced reasoning, calibrated scoring, and strong performance on memory retrieval benchmarks.

Findings

MemReranker-0.6B outperforms BGE-Reranker and matches larger models.

MemReranker-4B achieves 0.737 MAP, comparable to Gemini-3-Flash.

Models maintain efficiency and generalize well across domains.

Abstract

In agent memory systems, the reranking model serves as the critical bridge connecting user queries with long-term memory. Most systems adopt the "retrieve-then-rerank" two-stage paradigm, but generic reranking models rely on semantic similarity matching and lack genuine reasoning capabilities, leading to a problem where recalled results are semantically highly relevant yet do not contain the key information needed to answer the question. This deficiency manifests in memory scenarios as three specific problems. First, relevance scores are miscalibrated, making threshold-based filtering difficult. Second, ranking degrades when facing temporal constraints, causal reasoning, and other complex queries. Third, the model cannot leverage dialogue context for semantic disambiguation. This report introduces MemReranker, a reranking model family (0.6B/4B) built on Qwen3-Reranker through multi-stage LLM knowledge distillation. Multi-teacher pairwise comparisons generate calibrated soft labels, BCE pointwise distillation establishes well-distributed scores, and InfoNCE contrastive learning enhances hard-sample discrimination. Training data combines general corpora with memory-specific multi-turn dialogue data covering temporal constraints, causal reasoning, and coreference resolution. On the memory retrieval benchmark, MemReranker-0.6B substantially outperforms BGE-Reranker and matches open-source 4B/8B models as well as GPT-4o-mini on key metrics. MemReranker-4B further achieves 0.737 MAP, with several metrics on par with Gemini-3-Flash, while maintaining inference latency at only 10--20% of large models. On finance and healthcare vertical-domain benchmarks, the models preserve generalization capabilities on par with mainstream large-parameter rerankers.