Filtering the rejection set while preserving false discovery rate control

TL;DR

This paper introduces Focused BH, a method to filter rejection sets in multiple testing while maintaining false discovery rate control, improving interpretability in structured hypotheses.

Contribution

The paper presents a new filtering approach for multiple testing that guarantees FDR control under broad conditions, addressing interpretability issues in structured hypotheses.

Findings

Focused BH controls FDR under monotonicity and positive dependence.

Simulations show robust performance across diverse scenarios.

Real data analyses demonstrate practical utility in ICD and GO datasets.

Abstract

Scientific hypotheses in a variety of applications have domain-specific structures, such as the tree structure of the International Classification of Diseases (ICD), the directed acyclic graph structure of the Gene Ontology (GO), or the spatial structure in genome-wide association studies. In the context of multiple testing, the resulting relationships among hypotheses can create redundancies among rejections that hinder interpretability. This leads to the practice of filtering rejection sets obtained from multiple testing procedures, which may in turn invalidate their inferential guarantees. We propose Focused BH, a simple, flexible, and principled methodology to adjust for the application of any pre-specified filter. We prove that Focused BH controls the false discovery rate under various conditions, including when the filter satisfies an intuitive monotonicity property and the p-values are positively dependent. We demonstrate in simulations that Focused BH performs well across a variety of settings, and illustrate this method's practical utility via analyses of real datasets based on ICD and GO.