False discovery rate control with e-values

TL;DR

This paper introduces the e-BH procedure, a new method for controlling the false discovery rate using e-values, which are more flexible than p-values, especially under complex dependence structures.

Contribution

The paper develops the e-BH procedure, an analog of the BH method for e-values, and proves its FDR control under any dependence, expanding multiple testing tools.

Findings

e-BH controls FDR at the desired level without correction under dependence

e-BH is applicable in complex dependence, structured hypotheses, and multi-armed bandit problems

The standard BH procedure is a special case of e-BH via calibration

Abstract

E-values have gained attention as potential alternatives to p-values as measures of uncertainty, significance and evidence. In brief, e-values are realized by random variables with expectation at most one under the null; examples include betting scores, (point null) Bayes factors, likelihood ratios and stopped supermartingales. We design a natural analog of the Benjamini-Hochberg (BH) procedure for false discovery rate (FDR) control that utilizes e-values, called the e-BH procedure, and compare it with the standard procedure for p-values. One of our central results is that, unlike the usual BH procedure, the e-BH procedure controls the FDR at the desired level -- with no correction -- for any dependence structure between the e-values. We illustrate that the new procedure is convenient in various settings of complicated dependence, structured and post-selection hypotheses, and multi-armed bandit problems. Moreover, the BH procedure is a special case of the e-BH procedure through calibration between p-values and e-values. Overall, the e-BH procedure is a novel, powerful and general tool for multiple testing under dependence, that is complementary to the BH procedure, each being an appropriate choice in different applications.