Agentic-R: Learning to Retrieve for Agentic Search

TL;DR

Agentic-R introduces a novel retriever training framework for agentic search, optimizing passage utility with both local relevance and global correctness, and employs an iterative bidirectional training strategy to improve multi-turn reasoning in QA tasks.

Contribution

The paper proposes a new retriever training method tailored for agentic search, leveraging both local and global signals and an iterative training process for enhanced multi-turn question answering.

Findings

Outperforms strong baselines on seven QA benchmarks.

Effectively combines local relevance and global correctness in retrieval.

Iterative training improves retrieval quality over fixed-question methods.

Abstract

Agentic search has recently emerged as a powerful paradigm, where an agent interleaves multi-step reasoning with on-demand retrieval to solve complex questions. Despite its success, how to design a retriever for agentic search remains largely underexplored. Existing search agents typically rely on similarity-based retrievers, while similar passages are not always useful for final answer generation. In this paper, we propose a novel retriever training framework tailored for agentic search. Unlike retrievers designed for single-turn retrieval-augmented generation (RAG) that only rely on local passage utility, we propose to use both local query-passage relevance and global answer correctness to measure passage utility in a multi-turn agentic search. We further introduce an iterative training strategy, where the search agent and the retriever are optimized bidirectionally and iteratively. Different from RAG retrievers that are only trained once with fixed questions, our retriever is continuously improved using evolving and higher-quality queries from the agent. Extensive experiments on seven single-hop and multi-hop QA benchmarks demonstrate that our retriever, termed \ours{}, consistently outperforms strong baselines across different search agents. Our codes are available at: https://github.com/8421BCD/Agentic-R.