AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning

TL;DR

AutoSearch uses reinforcement learning to adaptively determine the optimal search depth in agentic RAG systems, balancing accuracy and efficiency for complex question answering.

Contribution

It introduces AutoSearch, a RL framework that dynamically evaluates search steps to minimize unnecessary search while maintaining high answer quality.

Findings

AutoSearch achieves better accuracy-efficiency trade-offs than fixed-depth methods.

AutoSearch reduces over-searching and computational costs in complex question answering.

Experiments show AutoSearch improves answer quality on multiple benchmarks.

Abstract

Agentic retrieval-augmented generation (RAG) systems enable large language models (LLMs) to solve complex tasks through multi-step interaction with external retrieval tools. However, such multi-step interaction often involves redundant search steps, incurring substantial computational cost and latency. Prior work limits search depth (i.e., the number of search steps) to reduce cost, but this often leads to underexploration of complex questions. To address this, we first investigate how search depth affects accuracy and find a minimal sufficient search depth that defines an accuracy-efficiency trade-off, jointly determined by question complexity and the agent's capability. Furthermore, we propose AutoSearch, a reinforcement learning (RL) framework that evaluates each search step via self-generated intermediate answers. By a self-answering mechanism, AutoSearch identifies the minimal sufficient search depth and promotes efficient search by rewarding its attainment while penalizing over-searching. In addition, reward mechanisms are introduced to stabilize search behavior and improve answer quality on complex questions. Extensive experiments on multiple benchmarks show that AutoSearch achieves a superior accuracy-efficiency trade-off, alleviating over-searching while preserving search quality.