Improving Wald's (approximate) sequential probability ratio test by avoiding overshoot

TL;DR

This paper enhances Wald's SPRT by modifying the test statistic to prevent overshoot, leading to improved power and guaranteed error control, especially in approximate and power-one scenarios, with extensions to confidence sequences and conformal martingales.

Contribution

It introduces a novel 'sequential boosting' method that reduces overshoot in SPRT, improving power and error guarantees in approximate and power-one tests.

Findings

Improved power-one SPRTs with less sample usage.

Guaranteed error control for approximate SPRT when eta > 0.

Extensions to confidence sequences and conformal martingales.

Abstract

Wald's sequential probability ratio test (SPRT) is a cornerstone of sequential analysis. Based on desired type-I, II error levels $\alpha, \beta$, it stops when the likelihood ratio crosses certain thresholds, guaranteeing optimality of the expected sample size. However, these thresholds are not closed form and the test is often applied with approximate thresholds $(1-\beta)/\alpha$ and $\beta/(1-\alpha)$ (approximate SPRT). When $\beta > 0$, this neither guarantees error control at $\alpha,\beta$ nor optimality. When $\beta=0$ (power-one SPRT), this method is conservative and not optimal. The looseness in both cases is caused by \emph{overshoot}: the test statistic overshoots the thresholds at the stopping time. Numerically calculating thresholds may be infeasible, and most software packages do not do this. We improve the approximate SPRT by modifying the test statistic to avoid overshoot. Our `sequential boosting' technique uniformly improves power-one SPRTs $(\beta=0)$ for simple nulls and alternatives, or for one-sided nulls and alternatives in exponential families. When $\beta > 0$, our techniques provide guaranteed error control at $\alpha,\beta$, while needing less samples than the approximate SPRT in our simulations. We also provide several nontrivial extensions: confidence sequences, sampling without replacement and conformal martingales.