Demystifying and Enhancing the Efficiency of Large Language Model Based Search Agents
Tiannuo Yang, Zebin Yao, Bowen Jin, Lixiao Cui, Yusen Li, Gang Wang, Xiaoguang Liu
TL;DR
This paper introduces SearchAgent-X, a high-efficiency framework for LLM-based search agents that significantly improves throughput and latency by optimizing retrieval methods and system design, enabling faster and more efficient complex task solving.
Contribution
The paper proposes SearchAgent-X, a novel inference framework that combines high-recall approximate retrieval with priority-aware scheduling and non-stall retrieval to enhance efficiency.
Findings
Achieves up to 3.4× higher throughput
Reduces latency by 5×
Outperforms state-of-the-art systems in diverse tasks
Abstract
Large Language Model (LLM)-based search agents have shown remarkable capabilities in solving complex tasks by dynamically decomposing problems and addressing them through interleaved reasoning and retrieval. However, this interleaved paradigm introduces substantial efficiency bottlenecks. First, we observe that both highly accurate and overly approximate retrieval methods degrade system efficiency: exact search incurs significant retrieval overhead, while coarse retrieval requires additional reasoning steps during generation. Second, we identify inefficiencies in system design, including improper scheduling and frequent retrieval stalls, which lead to cascading latency -- where even minor delays in retrieval amplify end-to-end inference time. To address these challenges, we introduce SearchAgent-X, a high-efficiency inference framework for LLM-based search agents. SearchAgent-X…
Peer Reviews
Decision·ICLR 2026 Poster
- The paper is well-written and easy to follow. - The paper shows the non-monotonic relationship between retrieval accuracy and efficiency. - Components (priority scheduling, non-stall retrieval) in SearchAgent-X are well-motivated. - Comprehensive ablation studies demonstrate robustness of the proposed method.
- The baselines do not include recent LLM serving framework like SGLang. - The priority discretization thresholds and non-stall retrieval maturity criterion involve heuristic settings without extensive sensitivity analysis.
1. With the rise of DeepResearch-related work, the paper systematically studies efficiency issues in this emerging scenario, offering significant practical significance and application value. 2. Through empirical studies, the paper discovers many noteworthy phenomena. It provides a deep analysis of the root causes of search agent latency, supported by extensive experiments. 3. Priority scheduling avoids complex weight tuning through discretization, while non-stall retrieval adaptively terminates
1. The main improvements stem from the high reusability of KV-cache, which works for the Search-R1 pattern. However, some current search agents modify historical context to handle overly long retrieval trajectories (e.g., summarizing previous context or keeping only recent k rounds of retrieval results), which could render the priority-aware scheduling completely ineffective. 2. Priority-aware scheduling cannot be applied to commercial models (e.g., GPT), and the non-stall retrieval mechanism ca
- The proposed method achieves good latency/throughput improvement while preserving the accuracy of search agents.
- No discussion about the previous efficiency optimization method for RAG with multiple rounds of retrievals, including [1-3] - The non-stall early termination strategy in Section 3.3 seems inconsistent with the retrieval-induced stalls in Section 2.2.2. It is unclear how this optimization resolves the stall caused by being late for the LLM generation steps. - The paper contains several unclear points. See the questions below. ### References - [1] https://arxiv.org/abs/2502.20969 - [2] https:/
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Multimodal Machine Learning Applications · Big Data and Digital Economy