The Bottom-Up Approach for Powerful Testing with FWER Control

TL;DR

This paper introduces a bottom-up method for multiple testing that optimizes power while controlling the family-wise error rate, resulting in more effective and computationally feasible procedures demonstrated through simulations and real data.

Contribution

It proposes a novel bottom-up approach for constructing consonant closed testing procedures that enhance power and computational efficiency in multiple testing.

Findings

Improved power over existing procedures in simulations.

Demonstrated effectiveness on real data.

Procedures are computationally practical.

Abstract

We seek to design novel multiple testing procedures, which take into account a relevant notion of ''power'' or true discovery on the one hand, and allow computationally efficient test design and application on the other. Towards this end we characterize the optimal procedures that strongly control the family-wise error rate, for a range of power objectives measuring the success of multiple testing procedures in making true individual discoveries, and under a reasonable set of assumptions. While we cannot generally find these optimal solutions in practice, we propose the bottom-up approach, which constructs consonant closed testing procedures, while taking into account the overall power objective in designing the tests on every level of the closed testing hierarchy. This leads to a general recipe, yielding novel procedures which are computationally practical and demonstrate substantially improved power in both simulations and a real data study, compared to existing procedures.