PeeriScope: A Multi-Faceted Framework for Evaluating Peer Review Quality

TL;DR

PeeriScope is a comprehensive, modular platform that assesses peer review quality using structured features, language model evaluations, and supervised predictions, supporting various stakeholders and promoting research in review quality.

Contribution

It introduces PeeriScope, a novel extensible framework combining multiple evaluation methods for peer review quality assessment.

Findings

Supports reviewer self-assessment, editorial triage, and auditing.

Provides a public interface and API for deployment and research.

Enables development of new review quality evaluation methods.

Abstract

The increasing scale and variability of peer review in scholarly venues has created an urgent need for systematic, interpretable, and extensible tools to assess review quality. We present PeeriScope, a modular platform that integrates structured features, rubric-guided large language model assessments, and supervised prediction to evaluate peer review quality along multiple dimensions. Designed for openness and integration, PeeriScope provides both a public interface and a documented API, supporting practical deployment and research extensibility. The demonstration illustrates its use for reviewer self-assessment, editorial triage, and large-scale auditing, and it enables the continued development of quality evaluation methods within scientific peer review. PeeriScope is available both as a live demo at https://app.reviewer.ly/app/peeriscope and via API services at https://github.com/Reviewerly-Inc/Peeriscope.