PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficient

TL;DR

This paper introduces PaperCompass, a hierarchical framework for small language-model paper-reading agents that improves efficiency and performance by separating high-level planning from detailed execution, using a novel RL method called DFPO.

Contribution

The paper proposes PaperCompass, a hierarchical approach inspired by cognitive science, and introduces DFPO, a new RL algorithm for training efficient paper-reading agents.

Findings

Achieves comparable performance to larger models on Paper-QA benchmarks.

Improves efficiency over strong baselines without performance loss.

Provides theoretical analysis supporting stable training.

Abstract

The accelerating growth of the scientific literature makes it increasingly difficult for researchers to track new advances through manual reading alone. Recent progress in large language models (LLMs) has therefore spurred interest in autonomous agents that can read scientific papers and extract task-relevant information. However, most existing approaches rely either on heavily engineered prompting or on a conventional SFT-RL training pipeline, both of which often lead to excessive and low-yield exploration. Drawing inspiration from cognitive science, we propose PaperCompass, a framework that mitigates these issues by separating high-level planning from fine-grained execution. PaperCompass first drafts an explicit plan that outlines the intended sequence of actions, and then performs detailed reasoning to instantiate each step by selecting the parameters for the corresponding function calls. To train such behavior, we introduce Draft-and-Follow Policy Optimization (DFPO), a tailored RL method that jointly optimizes both the draft plan and the final solution. DFPO can be viewed as a lightweight form of hierarchical reinforcement learning, aimed at narrowing the `knowing-doing' gap in LLMs. We provide a theoretical analysis that establishes DFPO's favorable optimization properties, supporting a stable and reliable training process. Experiments on paper-based question answering (Paper-QA) benchmarks show that PaperCompass improves efficiency over strong baselines without sacrificing performance, achieving results comparable to much larger models.