On a Reliable Peer-Review Process

TL;DR

This paper introduces a Bayesian-based peer-review process that incentivizes truthful review disclosures through a scoring system, aiming to improve review reliability and consensus accuracy.

Contribution

It presents a novel Bayesian model and scoring mechanism that promote honest reviews and facilitate consensus in peer review.

Findings

Reviewers maximize expected scores by truthful reporting

The scoring system encourages honest disclosures under mild assumptions

The method improves review reliability and consensus formation

Abstract

We propose an enhanced peer-review process where the reviewers are encouraged to truthfully disclose their reviews. We start by modelling that process using a Bayesian model where the uncertainty regarding the quality of the manuscript is taken into account. After that, we introduce a scoring function to evaluate the reported reviews. Under mild assumptions, we show that reviewers strictly maximize their expected scores by telling the truth. We also show how those scores can be used in order to reach consensus.