Asymptotic and compound e-values: multiple testing and empirical Bayes

TL;DR

This paper introduces and formalizes the concepts of compound p-values and e-values, demonstrating their role in controlling false discovery rates, connecting them to empirical Bayes methods, and providing new asymptotic constructions with theoretical guarantees.

Contribution

It explicitly defines compound e-values, shows their equivalence to FDR procedures, and develops asymptotic compound e-values for multiple t-tests with power improvements.

Findings

Compound e-values control FDR and can recover existing procedures.

Compound e-values are closed under averaging, enabling combination and derandomization.

Asymptotic compound e-values for t-tests offer power gains and theoretical guarantees.

Abstract

We explicitly define the notions of (bona fide, approximate or asymptotic) compound p-values and e-values, which have been implicitly presented and used in the recent multiple testing literature. While it is known that the e-BH procedure with compound e-values controls the FDR, we show the converse: every FDR controlling procedure can be recovered by instantiating the e-BH procedure with certain compound e-values. Since compound e-values are closed under averaging, this allows for combination and derandomization of arbitrary FDR procedures. We then connect compound e-values to empirical Bayes. In particular, we use the fundamental theorem of compound decision theory to derive the log-optimal simple separable compound e-value for testing a set of point nulls against point alternatives: it is a ratio of mixture likelihoods. As one example, we construct asymptotic compound e-values for multiple t-tests, where the (nuisance) variances may be different across hypotheses. Our construction may be interpreted as a data-driven instantiation of the optimal discovery procedure, and our results provide the first type-I error guarantees for the same, along with significant power gains.