Assessing replicability of findings across two studies of multiple features

TL;DR

This paper introduces new statistical procedures for assessing the replicability of findings across two independent studies, controlling error rates and improving power, with applications in genetics and microarray research.

Contribution

The paper proposes novel replicability analysis methods that incorporate null hypothesis fraction estimates to enhance power while controlling error rates.

Findings

The procedures control FWER and FDR for replicability claims.

Simulations show high power of the methods.

Real data examples demonstrate practical usefulness.

Abstract

Replicability analysis aims to identify the findings that replicated across independent studies that examine the same features. We provide powerful novel replicability analysis procedures for two studies for FWER and for FDR control on the replicability claims. The suggested procedures first select the promising features from each study solely based on that study, and then test for replicability only the features that were selected in both studies. We incorporate the plug-in estimates of the fraction of null hypotheses in one study among the selected hypotheses by the other study. Since the fraction of nulls in one study among the selected features from the other study is typically small, the power gain can be remarkable. We provide theoretical guarantees for the control of the appropriate error rates, as well as simulations that demonstrate the excellent power properties of the suggested procedures. We demonstrate the usefulness of our procedures on real data examples from two application fields: behavioural genetics and microarray studies.