PaperScout: An Autonomous Agent for Academic Paper Search with Process-Aware Sequence-Level Policy Optimization
Tingyue Pan, Jie Ouyang, Mingyue Cheng, Qingchuan Li, Zirui Liu, Daoyu Wang, Mingfan Pan, Shuo Yu, Qi Liu

TL;DR
PaperScout introduces an autonomous, process-aware agent for academic paper search that dynamically manages search actions, optimized with a novel sequence-level policy method, significantly improving retrieval performance over traditional workflows.
Contribution
The paper presents PaperScout, a novel autonomous agent framework for paper search, and Proximal Sequence Policy Optimization (PSPO), a new training method for multi-turn decision-making tasks.
Findings
Outperforms existing workflow-based and RL baselines in recall and relevance
Demonstrates effectiveness on synthetic and real-world benchmarks
Validates the benefits of adaptive, process-aware search strategies
Abstract
Academic paper search is a fundamental task in scientific research, yet most existing approaches rely on rigid, predefined workflows that struggle with complex, conditional queries. To address this limitation, we propose PaperScout, an autonomous agent that reformulates paper search as a sequential decision-making process. Unlike static workflows, PaperScout dynamically decides whether, when, and how to invoke search and expand tools based on accumulated retrieval context. However, training such agents presents a fundamental challenge: standard reinforcement learning methods, typically designed for single-turn tasks, suffer from a granularity mismatch when applied to multi-turn agentic tasks-where token-level optimization diverges from the granularity of sequence-level interactions-leading to noisy credit assignment and unstable training dynamics. We introduce Proximal Sequence Policy…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInformation Retrieval and Search Behavior · Machine Learning in Materials Science · Recommender Systems and Techniques
