The sequential rejection principle of familywise error control

TL;DR

This paper introduces the sequential rejection principle as a fundamental approach to familywise error control, unifying many existing methods and providing a new perspective for developing structured multiple testing procedures.

Contribution

It presents a general sequentially rejective testing procedure that encompasses many known methods and offers a theoretical foundation for error control in structured hypotheses.

Findings

Many well-known procedures are special cases of the proposed framework.

The sequential rejection principle ensures strong familywise error control under simple criteria.

The approach facilitates the development of procedures for graph-structured hypotheses.

Abstract

Closed testing and partitioning are recognized as fundamental principles of familywise error control. In this paper, we argue that sequential rejection can be considered equally fundamental as a general principle of multiple testing. We present a general sequentially rejective multiple testing procedure and show that many well-known familywise error controlling methods can be constructed as special cases of this procedure, among which are the procedures of Holm, Shaffer and Hochberg, parallel and serial gatekeeping procedures, modern procedures for multiple testing in graphs, resampling-based multiple testing procedures and even the closed testing and partitioning procedures themselves. We also give a general proof that sequentially rejective multiple testing procedures strongly control the familywise error if they fulfill simple criteria of monotonicity of the critical values and a limited form of weak familywise error control in each single step. The sequential rejection principle gives a novel theoretical perspective on many well-known multiple testing procedures, emphasizing the sequential aspect. Its main practical usefulness is for the development of multiple testing procedures for null hypotheses, possibly logically related, that are structured in a graph. We illustrate this by presenting a uniform improvement of a recently published procedure.