The Online Closure Principle

TL;DR

This paper introduces a rigorous framework and a new closure principle for online multiple testing, enabling control of error rates in sequential hypothesis testing scenarios where hypotheses arrive over time.

Contribution

It provides a formal definition of online multiple testing and develops an online closure principle that extends traditional methods to sequential, potentially infinite hypothesis testing.

Findings

Any online FWER controlling procedure can be derived from the online closure principle.

The paper proves admissibility of the proposed online closure procedures.

Short-cuts for online closed procedures are obtainable under consonance.

Abstract

The closure principle is fundamental in multiple testing and has been used to derive many efficient procedures with familywise error rate control. However, it is often unsuitable for modern research, which involves flexible multiple testing settings where not all hypotheses are known at the beginning of the evaluation. In this paper, we focus on online multiple testing where a possibly infinite sequence of hypotheses is tested over time. At each step, it must be decided on the current hypothesis without having any information about the hypotheses that have not been tested yet. Our main contribution is a general and stringent mathematical definition of online multiple testing and a new online closure principle which ensures that the resulting closed procedure can be applied in the online setting. We prove that any familywise error rate controlling online procedure can be derived by this online closure principle and provide admissibility results. In addition, we demonstrate how short-cuts of these online closed procedures can be obtained under a suitable consonance property.