Asymptotic Bayes-optimality under sparsity of some multiple testing procedures

TL;DR

This paper studies the asymptotic optimality of multiple testing procedures under sparsity, characterizing when popular methods like BH and Bonferroni are Bayes-optimal as the proportion of true effects diminishes.

Contribution

It provides a full characterization of fixed threshold rules that are asymptotically Bayes optimal under sparsity, including conditions for the optimality of BH and Bonferroni procedures.

Findings

Characterization of ABOS fixed threshold rules

Conditions for BFDR control procedures to be ABOS

Approximation of BH threshold by a nonrandom threshold

Abstract

Within a Bayesian decision theoretic framework we investigate some asymptotic optimality properties of a large class of multiple testing rules. A parametric setup is considered, in which observations come from a normal scale mixture model and the total loss is assumed to be the sum of losses for individual tests. Our model can be used for testing point null hypotheses, as well as to distinguish large signals from a multitude of very small effects. A rule is defined to be asymptotically Bayes optimal under sparsity (ABOS), if within our chosen asymptotic framework the ratio of its Bayes risk and that of the Bayes oracle (a rule which minimizes the Bayes risk) converges to one. Our main interest is in the asymptotic scheme where the proportion p of "true" alternatives converges to zero. We fully characterize the class of fixed threshold multiple testing rules which are ABOS, and hence derive conditions for the asymptotic optimality of rules controlling the Bayesian False Discovery Rate (BFDR). We finally provide conditions under which the popular Benjamini-Hochberg (BH) and Bonferroni procedures are ABOS and show that for a wide class of sparsity levels, the threshold of the former can be approximated by a nonrandom threshold.