Peerispect: Claim Verification in Scientific Peer Reviews

TL;DR

Peer review often includes subjective or unverified claims; Peerispect is an interactive system that automates claim verification by extracting claims, retrieving evidence, and verifying through natural language inference, aiding reviewers and committees.

Contribution

Introduces Peerispect, a modular, claim-level verification system for peer reviews, integrating extraction, retrieval, and verification with a visual interface and supporting multiple components.

Findings

Demonstrated Peerispect with a live demo and API services.

Enabled rapid inspection of claims with evidence highlighting.

Supported modular IR pipeline for flexible claim verification.

Abstract

Peer review is central to scientific publishing, yet reviewers frequently include claims that are subjective, rhetorical, or misaligned with the submitted work. Assessing whether review statements are factual and verifiable is crucial for fairness and accountability. At the scale of modern conferences and journals, manually inspecting the grounding of such claims is infeasible. We present Peerispect, an interactive system that operationalizes claim-level verification in peer reviews by extracting check-worthy claims from peer reviews, retrieving relevant evidence from the manuscript, and verifying the claims through natural language inference. Results are presented through a visual interface that highlights evidence directly in the paper, enabling rapid inspection and interpretation. Peerispect is designed as a modular Information Retrieval (IR) pipeline, supporting alternative retrievers, rerankers, and verifiers, and is intended for use by reviewers, authors, and program committees. We demonstrate Peerispect through a live, publicly available demo (https://app.reviewer.ly/app/peerispect) and API services (https://github.com/Reviewerly-Inc/Peerispect), accompanied by a video tutorial (https://www.youtube.com/watch?v=pc9RkvkUh14).