Online multiple testing with e-values

TL;DR

This paper introduces e-LOND, a new online multiple testing algorithm that controls the false discovery rate under arbitrary dependencies, enhancing power and applicability in continuous hypothesis testing scenarios.

Contribution

The paper presents e-LOND, an online FDR control method that works under unknown dependencies, and extends it for randomized testing and online confidence interval construction.

Findings

e-LOND controls FDR under arbitrary dependence.

e-LOND outperforms existing methods in simulations.

Extensions improve power and enable online confidence intervals.

Abstract

A scientist tests a continuous stream of hypotheses over time in the course of her investigation -- she does not test a predetermined, fixed number of hypotheses. The scientist wishes to make as many discoveries as possible while ensuring the number of false discoveries is controlled -- a well recognized way for accomplishing this is to control the false discovery rate (FDR). Prior methods for FDR control in the online setting have focused on formulating algorithms when specific dependency structures are assumed to exist between the test statistics of each hypothesis. However, in practice, these dependencies often cannot be known beforehand or tested after the fact. Our algorithm, e-LOND, provides FDR control under arbitrary, possibly unknown, dependence. We show that our method is more powerful than existing approaches to this problem through simulations. We also formulate extensions of this algorithm to utilize randomization for increased power, and for constructing confidence intervals in online selective inference.