DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking

TL;DR

DeepWideSearch introduces a new benchmark to evaluate agents' ability to perform both deep reasoning and wide-scale information retrieval, revealing current limitations and guiding future improvements in agent architectures.

Contribution

This paper presents DeepWideSearch, the first benchmark explicitly designed to evaluate the integration of depth and width in agentic information seeking tasks.

Findings

State-of-the-art agents achieve only 2.39% success rate on the benchmark.

Identified four key failure modes: lack of reflection, overreliance on internal knowledge, insufficient retrieval, context overflow.

Benchmark is publicly released to foster future research.

Abstract

Current search agents fundamentally lack the ability to simultaneously perform \textit{deep} reasoning over multi-hop retrieval and \textit{wide}-scale information collection-a critical deficiency for real-world applications like comprehensive market analysis and business development. To bridge this gap, we introduce DeepWideSearch, the first benchmark explicitly designed to evaluate agents to integrate depth and width in information seeking. In DeepWideSearch, agents must process a large volume of data, each requiring deep reasoning over multi-hop retrieval paths. Specifically, we propose two methods to converse established datasets, resulting in a curated collection of 220 questions spanning 15 diverse domains. Extensive experiments demonstrate that even state-of-the-art agents achieve only 2.39% average success rate on DeepWideSearch, highlighting the substantial challenge of integrating depth and width search in information-seeking tasks. Furthermore, our error analysis reveals four failure modes: lack of reflection, overreliance on internal knowledge, insufficient retrieval, and context overflow-exposing key limitations in current agent architectures. We publicly release DeepWideSearch to catalyze future research on more capable and robust information-seeking agents.