Conditional Equivalence Testing: an alternative remedy for publication bias

TL;DR

This paper proposes conditional equivalence testing (CET) as a new publication policy to reduce publication bias and improve reproducibility by allowing null results to be published through a two-stage testing process.

Contribution

It introduces CET as an alternative to traditional NHST, detailing its implementation, exploring its relation to Bayesian methods, and suggesting how journals can adopt it to mitigate publication bias.

Findings

CET can identify null effects that NHST might miss.

CET aligns with Bayesian testing in certain scenarios.

Implementation guidelines for CET in scientific publishing.

Abstract

We introduce a publication policy that incorporates conditional equivalence testing (CET), a two-stage testing scheme in which standard NHST is followed conditionally by testing for equivalence. The idea of CET is carefully considered as it has the potential to address recent concerns about reproducibility and the limited publication of null results. In this paper we detail the implementation of CET, investigate similarities with a Bayesian testing scheme, and outline the basis for how a scientific journal could proceed to reduce publication bias while remaining relevant.