CuSearch: Curriculum Rollout Sampling via Search Depth for Agentic RAG
Jianghan Shen, Siqi Luo, Xinyu Cheng, Jing Xiong, Yue Li, Jiyao Liu, Jiashi Lin, Yirong Chen, Junjun He

TL;DR
CuSearch introduces a curriculum sampling method that prioritizes deeper-search trajectories in RLVR-based agentic RAG training, leading to consistent performance improvements by leveraging search depth as a proxy for supervision density.
Contribution
The paper proposes CuSearch, a novel curriculum rollout sampling framework that dynamically reallocates training focus toward deeper trajectories based on search depth.
Findings
CuSearch improves performance by up to 11.8 exact-match points over standard methods.
Deeper-search trajectories provide denser supervision and are more informative for training.
Search depth serves as an effective, annotation-free proxy for supervision density in RLVR.
Abstract
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising paradigm for training agentic retrieval-augmented generation (RAG) systems from outcome-only supervision. Most existing methods optimize policies from uniformly sampled rollouts, implicitly treating all trajectories as equally informative. However, trajectories differ substantially in search depth and are therefore not equally informative: deeper-search trajectories contain more retrieval decision points and provide denser direct supervision for the retrieval sub-policy. Moreover, this heterogeneity grows over training as the within-batch depth distribution shifts toward higher values, yet uniform rollout sampling remains blind to this shift. To address this, we propose CuSearch, a curriculum rollout sampling framework built on Search-Depth Greedy Allocation (SDGA), a batch-level operator that reallocates a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
