The relative efficiency of sequential tests

TL;DR

This paper quantifies how sequential tests, specifically the Wald sequential probability ratio test, significantly reduce the average sample size needed compared to fixed sample size tests when testing a Brownian motion's drift, especially at higher power levels.

Contribution

It provides precise bounds on sample size reduction for sequential tests versus fixed tests, including symmetric and asymmetric error bounds, and shows the efficiency gains increase with test power.

Findings

Sequential tests reduce sample size by 36-75%.

Efficiency gains grow with higher test power.

Lower bounds established for asymmetric error bounds.

Abstract

While many statistical procedures rely on a fixed sample size, sequential methods allow a decision-maker to adapt the sample size to achieve a given precision. In this way, sequential tests reduce the average number of observations required to achieve a given power of the test -- but by how much? To address this question, we focus on the scenario of testing the unknown drift of a Brownian motion, comparing the Wald sequential probability ratio test with tests that use a pre-determined fixed sample size. We provide precise bounds on the average reduction in sample size needed to achieve a desired precision. Specifically, we demonstrate that for symmetric error bounds, the sequential test reduces the average sample size by at least 36\% and by at most 75\%. Moreover, the reduction in sample size increases monotonically with the power of the test, meaning that the relative advantage of using a sequential test over a fixed sample size test grows as higher power is required. We also study the relative efficiency in the case with asymmetric error bounds, and we provide a lower bound in terms of the symmetric case.