Analysis of error control in large scale two-stage multiple hypothesis testing

TL;DR

This paper proposes a two-stage multiple hypothesis testing procedure that effectively controls error rates while increasing power, by using an independent selection rule combined with the Bonferroni correction.

Contribution

It introduces a novel selection rule based on an independent statistic, enhancing error control and power in large-scale multiple testing scenarios.

Findings

The method controls Type 1 error rate asymptotically.

It reduces the multiple testing burden effectively.

Simulation results show higher power than traditional methods.

Abstract

When dealing with the problem of simultaneously testing a large number of null hypotheses, a natural testing strategy is to first reduce the number of tested hypotheses by some selection (screening or filtering) process, and then to simultaneously test the selected hypotheses. The main advantage of this strategy is to greatly reduce the severe effect of high dimensions. However, the first screening or selection stage must be properly accounted for in order to maintain some type of error control. In this paper, we will introduce a selection rule based on a selection statistic that is independent of the test statistic when the tested hypothesis is true. Combining this selection rule and the conventional Bonferroni procedure, we can develop a powerful and valid two-stage procedure. The introduced procedure has several nice properties: (i) it completely removes the selection effect; (ii) it reduces the multiplicity effect; (iii) it does not "waste" data while carrying out both selection and testing. Asymptotic power analysis and simulation studies illustrate that this proposed method can provide higher power compared to usual multiple testing methods while controlling the Type 1 error rate. Optimal selection thresholds are also derived based on our asymptotic analysis.