Empirical Bayes learning from selectively reported confidence intervals

TL;DR

This paper introduces a new empirical Bayes framework for analyzing selectively reported confidence intervals, addressing publication bias in biomedical literature and providing robust statistical inference with coverage guarantees.

Contribution

It extends empirical Bayes methods to handle truncated sampling due to publication bias, offering a novel selective tilting approach with theoretical guarantees.

Findings

Applied to 326,060 MEDLINE z-scores from 2000-2018

Provided coverage guarantees for posterior functionals

Addressed publication bias with a selective tilting method

Abstract

We develop a statistical framework for empirical Bayes learning from selectively reported confidence intervals, and apply it to provide context for interpreting results published in MEDLINE abstracts. We use a collection of 326,060 z-scores from MEDLINE abstracts (2000-2018) as the input for an empirical Bayes analysis, with publication bias as a key methodological challenge. We address publication bias through a selective tilting approach that extends empirical Bayes confidence intervals to truncated sampling. Our framework provides coverage guarantees for functionals including posterior estimands describing idealized replications and the symmetrized posterior mean, which we justify decision-theoretically as optimal among sign-equivariant (odd) estimators.