A covariate-adaptive test for replicability across multiple studies with false discovery rate control

TL;DR

This paper introduces ParFilter, a novel framework that improves power in large-scale replicability testing across multiple studies by combining filtering, covariate information, and false discovery rate control.

Contribution

ParFilter is a new method that enhances replicability analysis by reducing multiplicity burden and leveraging auxiliary covariates, with proven finite-sample FDR control.

Findings

ParFilter controls FDR under various dependence structures.

It outperforms existing methods in simulations.

Demonstrated effectiveness on autoimmunity RNA-Seq data.

Abstract

Replicability is a lynchpin for credible discoveries. The partial conjunction (PC) p-value, which combines individual base p-values from multiple similar studies, can gauge whether a feature of interest exhibits replicated signals across studies. However, when a large set of features are examined as in high-throughput experiments, testing for their replicated signals simultaneously can pose a very underpowered problem, due to both the multiplicity burden and inherent limitations of PC $p$-values. This power deficiency is markedly severe when replication is demanded for all studies under consideration, which is nonetheless the most natural and appealing benchmark for scientific generalizability a practitioner may request. We propose ParFilter, a general framework that marries the ideas of filtering and covariate-adaptiveness to power up large-scale testing for replicated signals as described above. It reduces the multiplicity burden by partitioning studies into smaller groups and borrowing the cross-group information to filter out unpromising features. Moreover, harnessing side information offered by auxiliary covariates whenever they are available, it can train informative hypothesis weights to encourage rejections of features more likely to exhibit replicated signals. We prove its finite-sample control on the false discovery rate, under both independence and arbitrary dependence among the base $p$-values across features. In simulations as well as a real case study on autoimmunity based on RNA-Seq data obtained from thymic cells, the ParFilter has demonstrated competitive performance against other existing methods for such replicability analyses.