HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches

TL;DR

HierSearch is a hierarchical deep search framework that effectively combines local and web searches using hierarchical reinforcement learning, improving multi-source retrieval accuracy across various domains.

Contribution

The paper introduces HierSearch, a hierarchical RL-based framework that integrates local and web search agents with a knowledge refiner, enhancing multi-source deep search capabilities.

Findings

Outperforms flat RL in deep search tasks.

Achieves superior results on six diverse benchmarks.

Effectively combines local and web searches with a knowledge refiner.

Abstract

Recently, large reasoning models have demonstrated strong mathematical and coding abilities, and deep search leverages their reasoning capabilities in challenging information retrieval tasks. Existing deep search works are generally limited to a single knowledge source, either local or the Web. However, enterprises often require private deep search systems that can leverage search tools over both local and the Web corpus. Simply training an agent equipped with multiple search tools using flat reinforcement learning (RL) is a straightforward idea, but it has problems such as low training data efficiency and poor mastery of complex tools. To address the above issue, we propose a hierarchical agentic deep search framework, HierSearch, trained with hierarchical RL. At the low level, a local deep search agent and a Web deep search agent are trained to retrieve evidence from their corresponding domains. At the high level, a planner agent coordinates low-level agents and provides the final answer. Moreover, to prevent direct answer copying and error propagation, we design a knowledge refiner that filters out hallucinations and irrelevant evidence returned by low-level agents. Experiments show that HierSearch achieves better performance compared to flat RL, and outperforms various deep search and multi-source retrieval-augmented generation baselines in six benchmarks across general, finance, and medical domains.