Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scaling
Xinglin Wang, Jiayi Shi, Shaoxiong Feng, Peiwen Yuan, Yiwei Li, Yueqi Zhang, Chuyi Tan, Ji Zhang, Boyuan Pan, Yao Hu, Kan Li

TL;DR
This paper introduces Recycling Search Experience (RSE), a novel test-time search strategy for large language models that recycles intermediate insights to improve efficiency and reasoning performance.
Contribution
RSE is a training-free, experience-guided approach that distills search trajectories into a shared bank, enabling positive and negative recycling to reduce redundant computations.
Findings
RSE outperforms strong baselines on multiple reasoning benchmarks.
Theoretically formalizes efficiency gains over independent sampling.
Achieves better compute-efficiency trade-offs in test-time scaling.
Abstract
Test-Time Scaling enhances the reasoning capabilities of Large Language Models by allocating additional inference compute to broaden the exploration of the solution space. However, existing search strategies typically treat rollouts as disposable samples, where valuable intermediate insights are effectively discarded after each trial. This wasted rollout-level experience leads to substantial computational redundancy, as models repeatedly re-derive discovered conclusions and revisit known dead ends across extensive attempts. To bridge this gap, we propose \textbf{Recycling Search Experience (RSE)}, a self-guided, training-free strategy that turns test-time search from a series of isolated trials into a cumulative, experience-guided process. By actively distilling raw trajectories into a shared experience bank, RSE enables positive recycling of intermediate conclusions to shortcut…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
