Closure properties of classes of multiple testing procedures

TL;DR

This paper examines the mathematical closure properties of classes of multiple testing procedures, especially step-up and step-down methods, under set operations like union, intersection, complement, and difference.

Contribution

It provides a detailed analysis of how monotonic and well-behaved multiple testing procedures behave under basic set operations, highlighting which classes are closed or not.

Findings

Union or intersection of arbitrary procedures yields monotonic but not well-behaved procedures.

Complement and difference operations generally do not preserve properties.

Step-up and step-down procedures are closed under union and intersection, but not under complement or difference.

Abstract

Statistical discoveries are often obtained through multiple hypothesis testing. A variety of procedures exists to evaluate multiple hypotheses, for instance the ones of Benjamini-Hochberg, Bonferroni, Holm or Sidak. We are particularly interested in multiple testing procedures with two desired properties: (solely) monotonic and well-behaved procedures. This article investigates to which extent the classes of (monotonic or well-behaved) multiple testing procedures, in particular the subclasses of so-called step-up and step-down procedures, are closed under basic set operations, specifically the union, intersection, difference and the complement of sets of rejected or non-rejected hypotheses. The present article proves two main results: First, taking the union or intersection of arbitrary (monotonic or well-behaved) multiple testing procedures results in new procedures which are monotonic but not well-behaved, whereas the complement or difference generally preserves neither property. Second, the two classes of (solely monotonic or well-behaved) step-up and step-down procedures are closed under taking the union or intersection, but not the complement or difference.