On Stepwise Control of the Generalized Familywise Error Rate

TL;DR

This paper introduces a unified framework for controlling the generalized familywise error rate ($k$-FWER) in multiple testing, extending classical procedures to allow for more false rejections while maintaining error control.

Contribution

It generalizes the closure principle and Hommel procedure for $k$-FWER control, providing a unified approach for stepwise multiple testing procedures.

Findings

Generalized closure principle for $k$-FWER control.

Equivalence of stepwise procedures and generalized closed testing.

Extension of Hommel procedure to $k$-FWER control.

Abstract

A classical approach for dealing with the multiple testing problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of at least one false rejection. In many applications, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of $k$ or more false rejections, which is called the $k$-FWER. In this article, a unified approach is presented for deriving the $k$-FWER controlling procedures. We first generalize the well-known closure principle in the context of the FWER to the case of controlling the $k$-FWER. Then, we discuss how to derive the $k$-FWER controlling stepwise (stepdown or stepup) procedures based on marginal $p$-values using this principle. We show that, under certain conditions, generalized closed testing procedures can be reduced to stepwise procedures, and any stepwise procedure is equivalent to a generalized closed testing procedure. Finally, we generalize the well-known Hommel procedure in two directions, and show that any generalized Hommel procedure is equivalent to a generalized closed testing procedure with the same critical values.