A unified class of null proportion estimators with plug-in FDR control

TL;DR

This paper introduces a unified class of null proportion estimators that enhance FDR control in multiple testing, allowing for flexible, distribution-aware, and plug-in adaptive procedures with proven guarantees.

Contribution

It proposes a comprehensive class of estimators unifying existing methods, with simple proofs of FDR control and the ability to incorporate distributional information, especially for discrete tests.

Findings

Unified estimators guarantee plug-in FDR control.

Convex combinations of estimators also control FDR.

Adaptations for discrete p-values improve performance.

Abstract

Since the work of \cite{Storey2004}, it is well-known that the performance of the Benjamini-Hochberg (BH) procedure can be improved by incorporating estimators of the number (or proportion) of null hypotheses, yielding an adaptive BH procedure which still controls FDR. Several such plug-in estimators have been proposed since then, for some of these, like Storey's estimator, plug-in FDR control has been established, while for some others, e.g. the estimator of \cite{PC2006}, some gaps remain to be closed. In this work we introduce a unified class of estimators, which encompasses existing and new estimators and unifies proofs of plug-in FDR control using simple convex ordering arguments. We also show that any convex combination of such estimators once more yields estimators with guaranteed plug-in FDR control. Additionally, the flexibility of the new class of estimators also allows incorporating distributional informations on the $p$-values. We illustrate this for the case of discrete tests, where the null distributions of the $p$-values are typically known. In that setting, we describe two generic approaches for adapting any estimator from the general class to the discrete setting while guaranteeing plug-in FDR control. While the focus of this paper is on presenting the generality and flexibility of the new class of estimators, we also include some analyses on simulated and real data.