EchoReview: Learning Peer Review from the Echoes of Scientific Citations

TL;DR

EchoReview introduces a novel citation-based data synthesis framework to train automated peer reviewers, leveraging collective evaluative signals from citations to improve review quality and scalability.

Contribution

It presents a new citation-context-driven data synthesis method and constructs the first large-scale review dataset, EchoReview-16K, to train a reliable automated reviewer model.

Findings

EchoReviewer-7B improves evidence support assessment.

Enhances review comprehensiveness and reliability.

Demonstrates citation context as effective data source.

Abstract

As the volume of scientific submissions continues to grow rapidly, traditional peer review systems are facing unprecedented scalability pressures, highlighting the urgent need for automated reviewing methods that are both scalable and reliable. Existing supervised fine-tuning approaches based on real review data are fundamentally constrained by single-source of data as well as the inherent subjectivity and inconsistency of human reviews, limiting their ability to support high-quality automated reviewers. To address these issues, we propose EchoReview, a citation-context-driven data synthesis framework that systematically mines implicit collective evaluative signals from academic citations and transforms scientific community's long-term judgments into structured review-style data. Based on this pipeline, we construct EchoReview-16K, the first large-scale, cross-conference, and cross-year citation-driven review dataset, and train an automated reviewer, EchoReviewer-7B. Experimental results demonstrate that EchoReviewer-7B can achieve significant and stable improvements on core review dimensions such as evidence support and review comprehensiveness, validating citation context as a robust and effective data paradigm for reliable automated peer review.