Type I error rate control for testing many hypotheses: a survey with proofs

TL;DR

This survey reviews recent methods for controlling type I error rates in multiple hypothesis testing, including new contributions like a binomial quantile-based procedure for FDP control under independence.

Contribution

It provides a unified overview of recent advances and introduces a novel binomial quantile-based method for FDP control under independence.

Findings

Unified results with simple proofs for kFWER and FDP control.

Introduction of a new binomial quantile-based FDP control procedure.

Demonstrated effectiveness under independence assumptions.

Abstract

This paper presents a survey on some recent advances for the type I error rate control in multiple testing methodology. We consider the problem of controlling the $k$-family-wise error rate (kFWER, probability to make $k$ false discoveries or more) and the false discovery proportion (FDP, proportion of false discoveries among the discoveries). The FDP is controlled either via its expectation, which is the so-called false discovery rate (FDR), or via its upper-tail distribution function. We aim at deriving general and unified results together with concise and simple mathematical proofs. Furthermore, while this paper is mainly meant to be a survey paper, some new contributions for controlling the kFWER and the upper-tail distribution function of the FDP are provided. In particular, we derive a new procedure based on the quantiles of the binomial distribution that controls the FDP under independence.