Anytime Validity is Free: Inducing Sequential Tests

TL;DR

This paper demonstrates that anytime valid sequential tests can match traditional tests in power without loss, and introduces methods for continuous testing and characterization of SPRT, enhancing flexibility in statistical inference.

Contribution

It shows how to construct anytime valid tests matching traditional tests after N observations and introduces a method for ongoing testing using conditional significance levels.

Findings

Anytime valid tests can match traditional tests in power after N observations.

A sequential testing procedure using outcome-based significance levels is valid at the original level.

The SPRT is characterized by invariance under test induction and linked to Neyman-Pearson tests for small significance levels.

Abstract

Anytime valid sequential tests permit us to stop testing based on the current data, without invalidating the inference. Given a maximum number of observations $N$, one may believe this must come at the cost of power when compared to a conventional test that waits until all $N$ observations have arrived. Our first contribution is to show that this is false: for any valid test based on $N$ observations, we show how to construct an anytime valid sequential test that matches it after $N$ observations. Our second contribution is that we may continue testing by using the outcome of a $[0, 1]$-valued test as a conditional significance level in subsequent testing, leading to an overall procedure that is valid at the original significance level. This shows that anytime validity and optional continuation are readily available in traditional testing, without requiring explicit use of e-values. We illustrate this by deriving the anytime valid sequentialized $z$-test and $t$-test, which at time $N$ coincide with the traditional $z$-test and $t$-test. Finally, we characterize the SPRT by invariance under test induction, and also show under an i.i.d. assumption that the SPRT is induced by the Neyman-Pearson test for a tiny significance level and huge $N$.