FACT: Fast closed testing for exchangeable local tests
Edgar Dobriban

TL;DR
The paper introduces FACT, an efficient algorithm for closed testing in multiple hypothesis testing that controls the family-wise error rate, applicable to various existing methods and capable of detecting both sparse and dense signals.
Contribution
It presents a quadratic-time exact algorithm for closed testing with symmetric, monotone tests, generalizing many existing methods and introducing the Simes-higher criticism fusion test.
Findings
Quadratic complexity in the number of tests for exact closure computation.
Effective detection of both sparse and dense signals.
Generalizes multiple existing closed testing procedures.
Abstract
Multiple hypothesis testing problems arise naturally in science. In this paper, we introduce the new Fast Closed Testing (FACT) method for multiple testing, controlling the family-wise error rate. This error rate is state of the art in many important application areas, and is preferred to false discovery rate control for many reasons, including that it leads to stronger reproducibility. The closure principle rejects an individual hypothesis if all global nulls of subsets containing it are rejected using some test statistics. It takes exponential time in the worst case. When the tests are symmetric and monotone, our method is an exact algorithm for computing the closure, quadratic in the number of tests, and linear in the number of discoveries. Our framework generalizes most examples of closed testing such as Holm's and the Bonferroni method. As a special case of our method, we propose…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Methods in Clinical Trials · Statistical Methods and Inference · Optimal Experimental Design Methods
