Asymptotically optimal, sequential, multiple testing procedures with prior information on the number of signals

TL;DR

This paper develops sequential multiple testing procedures that are asymptotically optimal and control error rates without distributional assumptions, leveraging prior information about the number of signals to improve efficiency.

Contribution

It introduces new procedures for multiple hypothesis testing with prior info on signal count, achieving asymptotic optimality and error control in a distribution-free setting.

Findings

Procedures control familywise error rates without distributional assumptions.

Knowing the exact number of signals halves the expected sample size.

Procedures are asymptotically optimal as error probabilities vanish.

Abstract

Assuming that data are collected sequentially from independent streams, we consider the simultaneous testing of multiple binary hypotheses under two general setups; when the number of signals (correct alternatives) is known in advance, and when we only have a lower and an upper bound for it. In each of these setups, we propose feasible procedures that control, without any distributional assumptions, the familywise error probabilities of both type I and type II below given, user-specified levels. Then, in the case of i.i.d. observations in each stream, we show that the proposed procedures achieve the optimal expected sample size, under every possible signal configuration, asymptotically as the two error probabilities vanish at arbitrary rates. A simulation study is presented in a completely symmetric case and supports insights obtained from our asymptotic results, such as the fact that knowledge of the exact number of signals roughly halves the expected number of observations compared to the case of no prior information.