Scaling Search-Augmented LLM Reasoning via Adaptive Information Control

TL;DR

This paper introduces DeepControl, a novel framework for adaptive information control in search-augmented reasoning agents, which improves reasoning performance by selectively regulating external information retrieval based on a formal utility measure.

Contribution

The paper proposes a formal utility-based approach for adaptive information control, including retrieval continuation and granularity mechanisms, to enhance reasoning agents' efficiency and effectiveness.

Findings

Achieves 9.4% and 8.6% performance improvements on two benchmarks.

Outperforms outcome-based RL baselines and retrieval-free methods.

Demonstrates the importance of adaptive information control in complex environments.

Abstract

Search-augmented reasoning agents interleave multi-step reasoning with external information retrieval, but uncontrolled retrieval often leads to redundant evidence, context saturation, and unstable learning. Existing approaches rely on outcome-based reinforcement learning (RL), which provides limited guidance for regulating information acquisition. We propose DeepControl, a framework for adaptive information control based on a formal notion of information utility, which measures the marginal value of retrieved evidence under a given reasoning state. Building on this utility, we introduce retrieval continuation and granularity control mechanisms that selectively regulate when to continue and stop retrieval, and how much information to expand. An annealed control strategy enables the agent to internalize effective information acquisition behaviors during training. Extensive experiments across seven benchmarks demonstrate that our method consistently outperforms strong baselines. In particular, our approach achieves average performance improvements of 9.4% and 8.6% on Qwen2.5-7B and Qwen2.5-3B, respectively, over strong outcome-based RL baselines, and consistently outperforms both retrieval-free and retrieval-based reasoning methods without explicit information control. These results highlight the importance of adaptive information control for scaling search-augmented reasoning agents to complex, real-world information environments.