UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seeking

TL;DR

This paper introduces UIS-Digger, a multi-agent framework designed to address the critical challenge of Unindexed Information Seeking (UIS), which involves retrieving vital information not captured by search engines, and provides a new benchmark for evaluating such systems.

Contribution

The paper presents UIS-QA, the first dedicated benchmark for UIS, and proposes UIS-Digger, a novel multi-agent system that effectively searches unindexed sources, setting a new baseline in the field.

Findings

State-of-the-art agents perform poorly on UIS-QA, highlighting the problem's severity.

UIS-Digger outperforms existing systems like O3 and GPT-4.1 on the benchmark.

Proactive interaction with unindexed sources improves information-seeking performance.

Abstract

Recent advancements in LLM-based information-seeking agents have achieved record-breaking performance on established benchmarks. However, these agents remain heavily reliant on search-engine-indexed knowledge, leaving a critical blind spot: Unindexed Information Seeking (UIS). This paper identifies and explores the UIS problem, where vital information is not captured by search engine crawlers, such as overlooked content, dynamic webpages, and embedded files. Despite its significance, UIS remains an underexplored challenge. To address this gap, we introduce UIS-QA, the first dedicated UIS benchmark, comprising 110 expert-annotated QA pairs. Notably, even state-of-the-art agents experience a drastic performance drop on UIS-QA (e.g., from 70.90 on GAIA and 46.70 on BrowseComp-zh to 24.55 on UIS-QA), underscoring the severity of the problem. To mitigate this, we propose UIS-Digger, a novel multi-agent framework that incorporates dual-mode browsing and enables simultaneous webpage searching and file parsing. With a relatively small $\sim$30B-parameter backbone LLM optimized using SFT and RFT training strategies, UIS-Digger sets a strong baseline at 27.27\%, outperforming systems integrating sophisticated LLMs such as O3 and GPT-4.1. This demonstrates the importance of proactive interaction with unindexed sources for effective and comprehensive information-seeking. Our work not only uncovers a fundamental limitation in current agent evaluation paradigms but also provides the first toolkit for advancing UIS research, defining a new and promising direction for robust information-seeking systems. The dataset has been released at: https://huggingface.co/datasets/UIS-Digger/UIS-QA.