Capturing the Severity of Type II Errors in High-Dimensional Multiple Testing

TL;DR

This paper develops a new multiple testing procedure that incorporates the severity of type II errors, aiming to improve decision accuracy in high-dimensional testing scenarios by minimizing false non-discoveries.

Contribution

It introduces a theoretically optimal multiple testing method that accounts for type II error severity under arbitrary dependence, enhancing decision-making accuracy.

Findings

The proposed procedure outperforms existing methods in simulations.

It is optimal in minimizing false non-discoveries given control of false discoveries.

Numerical evidence demonstrates its superior performance.

Abstract

The severity of type II errors is frequently ignored when deriving a multiple testing procedure, even though utilizing it properly can greatly help in making correct decisions. This paper puts forward a theory behind developing a multiple testing procedure that can incorporate the type II error severity and is optimal in the sense of minimizing a measure of false non-discoveries among all procedures controlling a measure of false discoveries. The theory is developed under a general model allowing arbitrary dependence by taking a compound decision theoretic approach to multiple testing with a loss function incorporating the type II error severity. We present this optimal procedure in its oracle form and offer numerical evidence of its superior performance over relevant competitors.