Simultaneous critical values for $t$-tests in very high dimensions

TL;DR

This paper introduces a data-driven method for determining simultaneous critical values for $t$-tests in high-dimensional settings, effectively controlling error rates like FDR and $k$-FWER, and demonstrating improved power over traditional $p$-value methods.

Contribution

It proposes a novel asymptotic procedure for setting critical values in multiple $t$-tests that requires minimal assumptions and includes a new estimator for the proportion of true alternatives.

Findings

Method controls FDR, $k$-FWER, and FDTP in high dimensions.

Simulation results show increased power compared to $p$-value based methods.

Application to leukemia microarray data demonstrates practical utility.

Abstract

This article considers the problem of multiple hypothesis testing using $t$-tests. The observed data are assumed to be independently generated conditional on an underlying and unknown two-state hidden model. We propose an asymptotically valid data-driven procedure to find critical values for rejection regions controlling the $k$-familywise error rate ($k$-FWER), false discovery rate (FDR) and the tail probability of false discovery proportion (FDTP) by using one-sample and two-sample $t$-statistics. We only require a finite fourth moment plus some very general conditions on the mean and variance of the population by virtue of the moderate deviations properties of $t$-statistics. A new consistent estimator for the proportion of alternative hypotheses is developed. Simulation studies support our theoretical results and demonstrate that the power of a multiple testing procedure can be substantially improved by using critical values directly, as opposed to the conventional $p$-value approach. Our method is applied in an analysis of the microarray data from a leukemia cancer study that involves testing a large number of hypotheses simultaneously.