Asymptotic Uncertainty of False Discovery Proportion

TL;DR

This paper investigates the asymptotic behavior and variance of the false discovery proportion (FDP) under weak dependence among test statistics, providing theoretical insights and practical recommendations for multiple testing procedures.

Contribution

It derives the asymptotic expansion of FDP and analyzes how its variance depends on dependence structures, offering new understanding and practical reporting suggestions.

Findings

Asymptotic expansion of FDP under mild conditions

Variance of FDP varies with dependence structure

Recommendation to report both mean and variance of FDP

Abstract

Multiple testing has been a popular topic in statistical research. Although vast works have been done, controlling the false discoveries remains a challenging task when the corresponding test statistics are dependent. Various methods have been proposed to estimate the false discovery proportion (FDP) under arbitrary dependence among the test statistics. One of the main ideas is to reduce arbitrary dependence to weak dependence and then to establish theoretically the strong consistency of the FDP and false discovery rate (FDR) under weak dependence. As a consequence, FDPs share the same asymptotic limit in the framework of weak dependence. We observe that the asymptotic variance of the FDP, however, may rely heavily on the dependence structure of the corresponding test statistics even when they are only weakly dependent; and it is of great practical value to quantify this variability, as it can serve as an indicator of the quality of the FDP estimate from the given data. As far as we are aware, the research on this respect is still limited in the literature. In this paper, we first derive the asymptotic expansion of FDP under mild regularity conditions and then examine how the asymptotic variance of FDP varies under different dependence structures both theoretically and numerically. With the observations in this study, we recommend that in a multiple testing performed by an FDP procedure, we may report both the mean and the variance estimates of FDP to enrich the study outcome.