Multiple Testing of Partial Conjunction Hypotheses for Assessing Replicability Across Dependent Studies

TL;DR

This paper introduces e-Filter, a new method for testing partial conjunction hypotheses that accounts for dependence among studies, improving replicability analysis especially in GWAS with sample overlap.

Contribution

The paper proposes e-Filter, a novel e-value based procedure that controls error rates under dependence, with proven validity and superior power over existing methods.

Findings

e-Filter controls FWER and FDR under dependence.

Simulation shows e-Filter has higher power than competitors.

Application to GWAS identifies more biologically relevant signals.

Abstract

Replicability is central to scientific progress, and the partial conjunction (PC) hypothesis testing framework provides an objective tool to quantify it across disciplines. Existing PC methods assume independent studies. Yet many modern applications, such as genome-wide association studies (GWAS) with sample overlap, violate this assumption, leading to dependence among study-specific summary statistics. Failure to account for this dependence can drastically inflate type I errors when combining inferences. We propose e-Filter, a powerful procedure grounded on the theory of e-values. It involves a filtering step that retains a set of the most promising PC hypotheses, and a selection step where PC hypotheses from the filtering step are marked as discoveries whenever their e-values exceed a selection threshold. We establish the validity of e-Filter for FWER and FDR control under unknown study dependence. A comprehensive simulation study demonstrates its excellent power gains over competing methods. We apply e-Filter to a GWAS replicability study to identify consistent genetic signals for low-density lipoprotein cholesterol (LDL-C). Here, the participating studies exhibit varying levels of sample overlap, rendering existing methods unsuitable for combining inferences. A subsequent pathway enrichment analysis shows that e-Filter replicated signals achieve stronger statistical enrichment on biologically relevant LDL-C pathways than competing approaches.