APEX-Searcher: Augmenting LLMs' Search Capabilities through Agentic Planning and Execution

TL;DR

APEX-Searcher enhances large language models' search abilities for complex multi-hop questions by decoupling planning and execution, using reinforcement learning and supervised fine-tuning to improve retrieval accuracy and reasoning.

Contribution

It introduces a two-stage agentic framework that separates planning and execution, addressing challenges in task reasoning and training for retrieval-augmented generation.

Findings

Significant improvements in multi-hop RAG performance.

Enhanced task planning accuracy across benchmarks.

Robust iterative sub-task execution capabilities.

Abstract

Retrieval-augmented generation (RAG), based on large language models (LLMs), serves as a vital approach to retrieving and leveraging external knowledge in various domain applications. When confronted with complex multi-hop questions, single-round retrieval is often insufficient for accurate reasoning and problem solving. To enhance search capabilities for complex tasks, most existing works integrate multi-round iterative retrieval with reasoning processes via end-to-end training. While these approaches significantly improve problem-solving performance, they are still faced with challenges in task reasoning and model training, especially ambiguous retrieval execution paths and sparse rewards in end-to-end reinforcement learning (RL) process, leading to inaccurate retrieval results and performance degradation. To address these issues, in this paper, we proposes APEX-Searcher, a novel Agentic Planning and Execution framework to augment LLM search capabilities. Specifically, we introduce a two-stage agentic framework that decouples the retrieval process into planning and execution: It first employs RL with decomposition-specific rewards to optimize strategic planning; Built on the sub-task decomposition, it then applies supervised fine-tuning on high-quality multi-hop trajectories to equip the model with robust iterative sub-task execution capabilities. Extensive experiments demonstrate that our proposed framework achieves significant improvements in both multi-hop RAG and task planning performances across multiple benchmarks.