Web2BigTable: A Bi-Level Multi-Agent LLM System for Internet-Scale Information Search and Extraction

TL;DR

Web2BigTable is a multi-agent system that enhances internet-scale information search and extraction by supporting both broad and deep reasoning tasks through a bi-level architecture and collaborative coordination.

Contribution

It introduces a novel bi-level multi-agent framework with a run--verify--reflect process and shared workspace for improved web-to-table search performance.

Findings

Achieved state-of-the-art results on WideSearch with 38.50 success rate.

Significantly outperformed previous methods in Row F1 and Item F1 metrics.

Successfully generalized to depth-oriented search with 73.0 accuracy.

Abstract

Agentic web search increasingly faces two distinct demands: deep reasoning over a single target, and structured aggregation across many entities and heterogeneous sources. Current systems struggle on both fronts. Breadth-oriented tasks demand schema-aligned outputs with wide coverage and cross-entity consistency, while depth-oriented tasks require coherent reasoning over long, branching search trajectories. We introduce \textbf{Web2BigTable}, a multi-agent framework for web-to-table search that supports both regimes. Web2BigTable adopts a bi-level architecture in which an upper-level orchestrator decomposes the task into sub-problems and lower-level worker agents solve them in parallel. Through a closed-loop run--verify--reflect process, the framework jointly improves decomposition and execution over time via persistent, human-readable external memory, with self-evolving updates to each single-agent. During execution, workers coordinate through a shared workspace that makes partial findings visible, allowing them to reduce redundant exploration, reconcile conflicting evidence, and adapt to emerging coverage gaps. Web2BigTable sets a new state of the art on WideSearch, reaching an Avg@4 Success Rate of \textbf{38.50} ($7.5\times$ the second best at 5.10), Row F1 of \textbf{63.53} (+25.03 over the second best), and Item F1 of \textbf{80.12} (+14.42 over the second best). It also generalises to depth-oriented search on XBench-DeepSearch, achieving 73.0 accuracy. Code is available at https://github.com/web2bigtable/web2bigtable.