A new multiple testing method in the dependent case

TL;DR

This paper introduces a novel multiple testing method that explicitly accounts for dependency among tests, improving accuracy and computational efficiency over traditional P-value based procedures, especially in large-scale testing scenarios.

Contribution

The new method incorporates dependency at all stages, offers an intuitive convexity property for admissibility, and demonstrates superior performance in simulations with dependent data.

Findings

Outperforms existing methods when the proportion of true positives is below 25%

Maintains computational feasibility for large numbers of tests

Shows improved control of false discoveries in dependent models

Abstract

The most popular multiple testing procedures are stepwise procedures based on $P$-values for individual test statistics. Included among these are the false discovery rate (FDR) controlling procedures of Benjamini--Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289--300] and their offsprings. Even for models that entail dependent data, $P$-values based on marginal distributions are used. Unlike such methods, the new method takes dependency into account at all stages. Furthermore, the $P$-value procedures often lack an intuitive convexity property, which is needed for admissibility. Still further, the new methodology is computationally feasible. If the number of tests is large and the proportion of true alternatives is less than say 25 percent, simulations demonstrate a clear preference for the new methodology. Applications are detailed for models such as testing treatments against control (or any intraclass correlation model), testing for change points and testing means when correlation is successive.