Coarse-to-fine Multiple Testing Strategies

TL;DR

This paper introduces a coarse-to-fine multiple testing strategy that leverages clustering of true hypotheses to improve power while controlling the familywise error rate in high-dimensional testing scenarios.

Contribution

It develops new procedures for Gaussian and non-parametric models that increase detection power by exploiting hypothesis clustering, surpassing Bonferroni bounds.

Findings

Higher power than Bonferroni estimates when hypotheses cluster

Effective control of FWER in high-dimensional settings

Addresses correlation issues between individual and cell-level tests

Abstract

We analyze control of the familywise error rate (FWER) in a multiple testing scenario with a great many null hypotheses about the distribution of a high-dimensional random variable among which only a very small fraction are false, or "active". In order to improve power relative to conservative Bonferroni bounds, we explore a coarse-to-fine procedure adapted to a situation in which tests are partitioned into subsets, or "cells", and active hypotheses tend to cluster within cells. We develop procedures for a standard linear model with Gaussian data and a non-parametric case based on generalized permutation testing, and demonstrate considerably higher power than Bonferroni estimates at the same FWER when the active hypotheses do cluster. The main technical difficulty arises from the correlation between the test statistics at the individual and cell levels, which increases the likelihood of a hypothesis being falsely discovered when the cell that contains it is falsely discovered (survivorship bias). This requires sharp estimates of certain quadrant probabilities when a cell is inactive.