Knowledge Integration Decay in Search-Augmented Reasoning of Large Language Models

TL;DR

This paper identifies a decay in knowledge integration during search-augmented reasoning in LLMs and proposes a simple, training-free method called SAKE to improve the retention and utilization of retrieved knowledge, enhancing reasoning performance.

Contribution

The paper introduces SAKE, a novel inference-time technique that stabilizes knowledge integration in LLMs, addressing the underexplored Knowledge Integration Decay problem.

Findings

SAKE significantly reduces knowledge decay in LLM reasoning.

Improves performance on multi-hop QA and complex reasoning tasks.

A lightweight, training-free approach enhances knowledge retention.

Abstract

Modern Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks by employing search-augmented reasoning to incorporate external knowledge into long chains of thought. However, we identify a critical yet underexplored bottleneck in this paradigm, termed Knowledge Integration Decay (KID). Specifically, we observe that as the length of reasoning generated before search grows, models increasingly fail to integrate retrieved evidence into subsequent reasoning steps, limiting performance even when relevant information is available. To address this, we propose Self-Anchored Knowledge Encoding (SAKE), a training-free inference-time strategy designed to stabilize knowledge utilization. By anchoring retrieved knowledge at both the beginning and end of the reasoning process, SAKE prevents it from being overshadowed by prior context, thereby preserving its semantic integrity. Extensive experiments on multi-hop QA and complex reasoning benchmarks demonstrate that SAKE significantly mitigates KID and improves performance, offering a lightweight yet effective solution for knowledge integration in agentic LLMs.