The edge of discovery: Controlling the local false discovery rate at the margin

TL;DR

This paper introduces a simple multiple testing method that controls the maximum local false discovery rate without prior knowledge, effectively balancing false positives and negatives in large-scale testing.

Contribution

It proposes a new procedure that controls the maximum lfdr across rejections and asymptotically matches the oracle Bayes approach without prior distribution knowledge.

Findings

Controls the expectation of the maximum lfdr across rejections

Asymptotically implements the oracle Bayes procedure

Derives the limiting distribution of maximum lfdr and empirical Bayes regret

Abstract

Despite the popularity of the false discovery rate (FDR) as an error control metric for large-scale multiple testing, its close Bayesian counterpart the local false discovery rate (lfdr), defined as the posterior probability that a particular null hypothesis is false, is a more directly relevant standard for justifying and interpreting individual rejections. However, the lfdr is difficult to work with in small samples, as the prior distribution is typically unknown. We propose a simple multiple testing procedure and prove that it controls the expectation of the maximum lfdr across all rejections; equivalently, it controls the probability that the rejection with the largest p-value is a false discovery. Our method operates without knowledge of the prior, assuming only that the p-value density is uniform under the null and decreasing under the alternative. We also show that our method asymptotically implements the oracle Bayes procedure for a weighted classification risk, optimally trading off between false positives and false negatives. We derive the limiting distribution of the attained maximum lfdr over the rejections, and the limiting empirical Bayes regret relative to the oracle procedure.