SPAR: Scholar Paper Retrieval with LLM-based Agents for Enhanced Academic Search

TL;DR

SPAR is a novel multi-agent framework utilizing LLMs for more flexible academic literature search, significantly improving retrieval performance and introducing a new benchmark for evaluation.

Contribution

It introduces SPAR, a multi-agent system with query decomposition and evolution, and SPARBench, a new benchmark with expert relevance labels, advancing scholarly retrieval methods.

Findings

SPAR outperforms baselines with up to +56% F1 on AutoScholar.

SPAR achieves up to +23% F1 on SPARBench.

The framework enhances search flexibility and effectiveness.

Abstract

Recent advances in large language models (LLMs) have opened new opportunities for academic literature retrieval. However, existing systems often rely on rigid pipelines and exhibit limited reasoning capabilities. We introduce SPAR, a multi-agent framework that incorporates RefChain-based query decomposition and query evolution to enable more flexible and effective search. To facilitate systematic evaluation, we also construct SPARBench, a challenging benchmark with expert-annotated relevance labels. Experimental results demonstrate that SPAR substantially outperforms strong baselines, achieving up to +56% F1 on AutoScholar and +23% F1 on SPARBench over the best-performing baseline. Together, SPAR and SPARBench provide a scalable, interpretable, and high-performing foundation for advancing research in scholarly retrieval. Code and data will be available at: https://github.com/xiaofengShi/SPAR