# Only Closed Testing Procedures are Admissible for Controlling False Discovery Proportions

## TL;DR

This paper proves that all methods controlling false discovery proportions are essentially equivalent to closed testing procedures or can be improved by them, simplifying the design of such statistical methods.

## Contribution

It establishes that only closed testing procedures are admissible for controlling false discovery proportions, unifying and simplifying the design of multiple testing methods.

## Key findings

- All such methods are either equivalent to or improved by closed testing methods.
- A closed testing method is admissible if and only if all local tests are admissible.
- Constructed a uniform improvement of a recent method demonstrating practical usefulness.

## Abstract

We consider the class of all multiple testing methods controlling tail probabilities of the false discovery proportion, either for one random set or simultaneously for many such sets. This class encompasses methods controlling familywise error rate, generalized familywise error rate, false discovery exceedance, joint error rate, simultaneous control of all false discovery proportions, and others, as well as seemingly unrelated methods such as gene set testing in genomics and cluster inference methods in neuroimaging. We show that all such methods are either equivalent to a closed testing method, or are uniformly improved by one. Moreover, we show that a closed testing method is admissible as a method controlling tail probabilities of false discovery proportions if and only if all its local tests are admissible. This implies that, when designing such methods, it is sufficient to restrict attention to closed testing methods only. We demonstrate the practical usefulness of this design principle by constructing a uniform improvement of a recently proposed method.