A unified treatment of multiple testing with prior knowledge using the p-filter

TL;DR

The paper introduces p-filter, a unified algorithmic framework that incorporates various types of prior knowledge into multiple testing procedures to improve power and control FDR.

Contribution

It presents a novel, unified framework that simultaneously integrates multiple forms of prior knowledge into multiple testing, recovering existing algorithms as special cases.

Findings

p-filter effectively combines prior knowledge types (a)-(d).

It improves testing power while controlling FDR.

The framework generalizes several existing methods.

Abstract

There is a significant literature on methods for incorporating knowledge into multiple testing procedures so as to improve their power and precision. Some common forms of prior knowledge include (a) beliefs about which hypotheses are null, modeled by non-uniform prior weights; (b) differing importances of hypotheses, modeled by differing penalties for false discoveries; (c) multiple arbitrary partitions of the hypotheses into (possibly overlapping) groups; and (d) knowledge of independence, positive or arbitrary dependence between hypotheses or groups, suggesting the use of more aggressive or conservative procedures. We present a unified algorithmic framework called p-filter for global null testing and false discovery rate (FDR) control that allows the scientist to incorporate all four types of prior knowledge (a)-(d) simultaneously, recovering a variety of known algorithms as special cases.