A New Step-down Procedure for Simultaneous Hypothesis Testing Under Dependence

TL;DR

This paper introduces a novel step-down multiple hypothesis testing procedure that accounts for dependence among test statistics, providing a Bayesian framework with a decision-theoretic justification and demonstrating superior performance in simulations.

Contribution

It develops a new dependence-aware step-down testing method with a Bayesian approach and decision-theoretic admissibility, applicable to both normal and non-normal data.

Findings

Outperforms existing procedures in misclassification rates across various dependence structures.

Provides computational simplifications through an alternative test statistic representation.

Demonstrates effectiveness in simulations with different sparsity levels.

Abstract

In this article, we consider the problem of simultaneous testing of hypotheses when the individual test statistics are not necessarily independent. Specifically, we consider the problem of simultaneous testing of point null hypotheses against two-sided alternatives for the mean parameters of normally distributed random variables. We assume that conditionally given the vector of means, these random variables jointly follow a multivariate normal distribution with a known but arbitrary covariance matrix. We consider a Bayesian framework where each unknown mean parameter is modeled through a two-component "spike and slab" mixture prior. This way, unconditionally the test statistics jointly have a mixture of multivariate normal distributions. A new testing procedure is developed that uses the dependence among the test statistics and works in a "step-down" manner. This procedure is general enough to be applied for non-normal data. A decision theoretic justification in favor of the proposed testing procedure has been provided by showing that unlike many traditional p-value based stepwise procedures, this new method possesses a certain "convexity property" which makes it admissible with respect to a vector risk function that captures the risks for the individual testing problems. An alternative representation of the proposed test statistics has also been established resulting in great simplification in the computational complexity. It is demonstrated through extensive simulations that for various forms of dependence and a wide range of sparsity levels, the proposed testing procedure compares quite favorably with several existing multiple testing procedures available in the literature in terms of overall misclassification probability.