An adaptive procedure for detecting replicated signals with $k$-family-wise error rate control

TL;DR

This paper introduces AdaFilter-AdaBon, an improved adaptive procedure for detecting replicated signals that enhances statistical power in high-throughput meta-analyses while maintaining control over the k-family-wise error rate.

Contribution

It develops a new method that incorporates a post-filter null proportion estimate to reduce conservativeness and improve power over existing procedures for replicability testing.

Findings

Demonstrates asymptotic k-FWER control under weak dependence.

Shows empirical finite-sample control with higher power.

Outperforms AdaFilter-Bon in simulation studies.

Abstract

Partial conjunction (PC) hypothesis testing is widely used to assess the replicability of scientific findings across multiple comparable studies. In high-throughput meta-analyses, testing a large number of PC hypotheses with k-family-wise error rate (k-FWER) control often suffers from low statistical power due to the multiplicity burden. The state-of-the-art AdaFilter-Bon procedure by Wang et al. (2022, Ann. Stat., 50(4), 1890-1909) alleviates this problem by filtering out hypotheses unlikely to be false before applying a rejection rule. However, a side effect of filtering is that it renders the rejection rule more stringent than necessary, leading to conservative k-FWER control. In this paper, we mitigate this conservativeness - and thereby improve the power of AdaFilter-Bon - by incorporating a post-filter null proportion estimate into the procedure. The resulting method, AdaFilter-AdaBon, has proven asymptotic k-FWER control under weak dependence and demonstrates empirical finite-sample control with higher power than the original AdaFilter-Bon in simulations.