The replication of equivalence studies

TL;DR

This paper introduces two new statistical methods, the two-trials rule and sceptical TOST, for analyzing equivalence replication studies, addressing a gap in methodology for non-superiority designs.

Contribution

It develops and adapts two methods for equivalence replication studies, enabling better assessment of reproducibility beyond superiority tests.

Findings

Both methods control Type-I error rate.

Sceptical TOST allows success even with non-significant results.

Methods facilitate sample size planning for replications.

Abstract

Replication studies are increasingly conducted to assess the credibility of scientific findings. Most of these replication attempts target studies with a superiority design, but there is a lack of methodology regarding the analysis of replication studies with alternative types of designs, such as equivalence. In order to fill this gap, we propose two approaches, the two-trials rule and the sceptical TOST procedure, adapted from methods used in superiority settings. Both methods have the same overall Type-I error rate, but the sceptical TOST procedure allows replication success even for non-significant original or replication studies. This leads to a larger project power and other differences in relevant operating characteristics. Both methods can be used for sample size calculation of the replication study, based on the results from the original one. The two methods are applied to data from the Reproducibility Project: Cancer Biology.