MMCTest - A Safe Algorithm for Implementing Multiple Monte Carlo Tests
Axel Gandy, Georg Hahn

TL;DR
MMCTest is a sequential algorithm that safely implements multiple hypothesis tests using simulation, providing high-probability correctness and efficiency, applicable to procedures like FDR and Bonferroni.
Contribution
Introduces MMCTest, a novel sequential method for multiple testing with simulation, ensuring high-probability correctness and computational efficiency.
Findings
Effective in biological data scenarios
Computationally more efficient than existing methods
Provides reliable decision sets at any interruption point
Abstract
Consider testing multiple hypotheses using tests that can only be evaluated by simulation, such as permutation tests or bootstrap tests. This article introduces MMCTest, a sequential algorithm which gives, with arbitrarily high probability, the same classification as a specific multiple testing procedure applied to ideal p-values. The method can be used with a class of multiple testing procedures which includes the Benjamini & Hochberg False Discovery Rate (FDR) procedure and the Bonferroni correction controlling the Familywise Error Rate. One of the key features of the algorithm is that it stops sampling for all the hypotheses which can already be decided as being rejected or non-rejected. MMCTest can be interrupted at any stage and then returns three sets of hypotheses: the rejected, the non-rejected and the undecided hypotheses. A simulation study motivated by actual biological data…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
