Identification of and correction for publication bias

TL;DR

This paper addresses publication bias by proposing methods to identify and correct for selective publication, improving the accuracy of meta-analyses and replication studies in social sciences.

Contribution

It introduces two novel approaches for estimating publication probabilities and provides unbiased estimators to correct for publication bias.

Findings

Effective identification of publication bias in large-scale studies

Median-unbiased estimators successfully correct for bias

Application to economics and psychology demonstrates practical utility

Abstract

Some empirical results are more likely to be published than others. Such selective publication leads to biased estimates and distorted inference. This paper proposes two approaches for identifying the conditional probability of publication as a function of a study's results, the first based on systematic replication studies and the second based on meta-studies. For known conditional publication probabilities, we propose median-unbiased estimators and associated confidence sets that correct for selective publication. We apply our methods to recent large-scale replication studies in experimental economics and psychology, and to meta-studies of the effects of minimum wages and de-worming programs.