A glass half full interpretation of the replicability of psychological science

TL;DR

This paper offers a nuanced interpretation of psychological science replicability, highlighting that a majority of replications are consistent with original findings when considering prediction intervals, thus portraying a more optimistic view of scientific robustness.

Contribution

It provides a reanalysis of the Reproducibility Project: Psychology, demonstrating that 77% of effect sizes fall within expected prediction intervals, challenging the simplistic narrative of low replicability.

Findings

77% of replication effect sizes within prediction intervals

Reproducibility Project results are more positive than media reports suggest

A more nuanced understanding of replicability in psychology

Abstract

A recent study of the replicability of key psychological findings is a major contribution toward understanding the human side of the scientific process. Despite the careful and nuanced analysis reported in the paper, mass and social media adhered to the simple narrative that only 36% of the studies replicated their original results. Here we show that 77% of the replication effect sizes reported were within a prediction interval based on the original effect size. In this light, the results of Reproducibility Project: Psychology can be viewed as a positive result for the scientific process.