On the expected runtime of multiple testing algorithms with bounded error

TL;DR

This paper analyzes the expected runtime of multiple testing algorithms that rely on Monte Carlo p-value approximations, providing finite and infinite expected runtime results under correctness guarantees.

Contribution

It offers new theoretical insights into the expected runtime behavior of algorithms with confidence guarantees in multiple hypothesis testing.

Findings

Finite expected runtime results for certain algorithms

Infinite expected runtime scenarios identified

Theoretical bounds on runtime under various conditions

Abstract

Consider testing multiple hypotheses in the setting where the p-values of all hypotheses are unknown and thus have to be approximated using Monte Carlo simulations. One class of algorithms published in the literature for this scenario provides guarantees on the correctness of their testing result through the computation of confidence statements on all approximated p-values. This article focuses on the expected runtime of such algorithms and derives a variety of finite and infinite expected runtime results.