DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning

TL;DR

DeepDiver introduces a reinforcement learning framework that enables large language models to adaptively scale their search efforts on the open web, significantly improving information seeking capabilities beyond traditional fixed prompting methods.

Contribution

It presents WebPuzzle, a new benchmark for open-web question answering, and develops DeepDiver, an RL-based approach that enhances search intensity scaling in LLMs, outperforming existing models on real-world tasks.

Findings

DeepDiver achieves comparable performance to much larger models on open-web tasks.

The search policy generalizes from closed-ended to open-ended queries.

WebPuzzle provides a rigorous benchmark for evaluating open-web information seeking.

Abstract

Information seeking demands iterative evidence gathering and reflective reasoning, yet large language models (LLMs) still struggle with it in open-web question answering. Existing prompting and supervised fine-tuning (SFT) methods remain fixed by prompt rules or training corpora, and are usually benchmarked only on well-structured wiki sources, limiting real-world adaptability. We introduce WebPuzzle, a 24k-sample training and 275-sample test benchmark that evaluates information seeking on the live internet, across both wiki and open-domain queries. Leveraging 7k WebPuzzle instances, we develop DeepDiver, a reinforcement-learning (RL) framework that cultivates Search Intensity Scaling (SIS)-an emergent ability to escalate search frequency and depth instead of settling on overconfident, under-evidenced answers. With SIS, Qwen2.5-7B-Instruct and Pangu-7B-Reasoner attain performance on real-web tasks comparable to the 671B-parameter DeepSeek-R1. We detail DeepDiver's curriculum from cold-start SFT to a well designed RL procedure, and show that its seeking policy generalized from closed-ended queries to open-ended generation such as long-form writing. Our results advance adaptive information seeking in LLMs and provide a rigorous benchmark for future work.