Dynamic adaptive multiple tests with finite sample FDR control

TL;DR

This paper develops new finite-sample adaptive multiple testing procedures that estimate the number of true null hypotheses to control the false discovery rate, including formulas, estimators, and dynamic stepwise tests.

Contribution

It introduces a class of adaptive tests with exact FDR control based on generalized Storey estimators and dynamic weighting, extending existing methods.

Findings

Derived exact FDR formulas for adaptive tests

Introduced generalized Storey estimators and weighted versions

Provided a converse to the Benjamini-Hochberg theorem

Abstract

The present paper introduces new adaptive multiple tests which rely on the estimation of the number of true null hypotheses and which control the false discovery rate (FDR) at level alpha for finite sample size. We derive exact formulas for the FDR for a large class of adaptive multiple tests which apply to a new class of testing procedures. In the following, generalized Storey estimators and weighted versions are introduced and it turns out that the corresponding adaptive step up and step down tests control the FDR. The present results also include particular dynamic adaptive step wise tests which use a data dependent weighting of the new generalized Storey estimators. In addition, a converse of the Benjamini Hochberg (1995) theorem is given. The Benjamini Hochberg (1995) test is the only "distribution free" step up test with FDR independent of the distribution of the p-values of false null hypotheses.