Why ex post peer review encourages high-risk research while ex ante review discourages it

TL;DR

This paper analyzes how ex ante and ex post peer review influence scientists to pursue different types of research questions, with ex post review encouraging high-risk, surprising results, and ex ante review shaping research based on reviewer expectations.

Contribution

It introduces a theoretical framework comparing ex ante and ex post peer review, revealing how each mode influences research choices and scientific innovation.

Findings

Ex post review promotes high-risk, surprising research.

Ex ante review influences research based on reviewer beliefs.

Different review modes lead to distinct scientific question selection.

Abstract

Peer review is an integral component of contemporary science. While peer review focuses attention on promising and interesting science, it also encourages scientists to pursue some questions at the expense of others. Here, we use ideas from forecasting assessment to examine how two modes of peer review -- ex ante review of proposals for future work and ex post review of completed science -- motivate scientists to favor some questions instead of others. Our main result is that ex ante and ex post peer review push investigators toward distinct sets of scientific questions. This tension arises because ex post review allows an investigator to leverage her own scientific beliefs to generate results that others will find surprising, whereas ex ante review does not. Moreover, ex ante review will favor different research questions depending on whether reviewers rank proposals in anticipation of changes to their own personal beliefs, or to the beliefs of their peers. The tension between ex ante and ex post review puts investigators in a bind, because most researchers need to find projects that will survive both. By unpacking the tension between these two modes of review, we can understand how they shape the landscape of science and how changes to peer review might shift scientific activity in unforeseen directions.