A frequentist local false discovery rate

TL;DR

This paper introduces a frequentist version of the local false discovery rate (lfdr) that estimates the probability of a null hypothesis being true at each point without relying on Bayesian priors, enabling more precise multiple testing decisions.

Contribution

It defines a frequentist lfdr that retains key properties of the Bayesian lfdr, providing a calibrated, prior-free measure for individual hypotheses and optimal rejection rules.

Findings

The frequentist lfdr is well-defined for continuous test statistics.

Thresholding the lfdr optimizes the separation of true and false nulls.

The method can be efficiently estimated and controlled under independence.

Abstract

The local false discovery rate (lfdr) of Efron et al. (2001) enjoys major conceptual and decision-theoretic advantages over the false discovery rate (FDR) as an error criterion in multiple testing, but is only well-defined in Bayesian models where the truth status of each null hypothesis is random. We define a frequentist counterpart to the lfdr based on the relative frequency of nulls at each point in the sample space. The frequentist lfdr is defined without reference to any prior, but preserves several important properties of the Bayesian lfdr: For continuous test statistics, $\text{lfdr}(t)$ gives the probability, conditional on observing some statistic equal to $t$, that the corresponding null hypothesis is true. Evaluating the lfdr at an individual test statistic also yields a calibrated forecast of whether its null hypothesis is true. Finally, thresholding the lfdr at $\frac{1}{1+\lambda}$ gives the best separable rejection rule under the weighted classification loss where Type I errors are $\lambda$ times as costly as Type II errors. The lfdr can be estimated efficiently using parametric or non-parametric methods, and a closely related error criterion can be provably controlled in finite samples under independence assumptions. Whereas the FDR measures the average quality of all discoveries in a given rejection region, our lfdr measures how the quality of discoveries varies across the rejection region, allowing for a more fine-grained analysis without requiring the introduction of a prior.