Average Power and $\lambda$-power in Multiple Testing Scenarios when the Benjamini-Hochberg False Discovery Rate Procedure is Used

TL;DR

This paper analyzes the power and false discovery rate control in multiple testing using the Benjamini-Hochberg procedure, providing theoretical results and simulations to guide study design in genomics and biomarker research.

Contribution

It introduces new asymptotic and finite-sample results for average power and $ extit{ extlambda}$-power, including central limit theorems for key quantities in multiple testing.

Findings

Proves strong consistency and asymptotic normality for key testing metrics.

Provides approximate formulas for $ extlambda$-power and FDR control.

Includes extensive simulation results for practical study design.

Abstract

We discuss several approaches to defining power in studies designed around the Benjamini-Hochberg (BH) false discovery rate (FDR) procedure. We focus primarily on the \textit{average power} and the $\lambda$-\textit{power}, which are the expected true positive fraction and the probability that the true positive fraction exceeds $\lambda$, respectively. We prove results concerning strong consistency and asymptotic normality for the positive call fraction (PCF), the true positive fraction (TPF) and false discovery fraction (FDF). Convergence of their corresponding expected values, including a convergence result for the average power, follow as a corollaries. After reviewing what is known about convergence in distribution of the errors of the plugin procedure, (Genovese, 2004), we prove central limit theorems for fully empirical versions of the PCF, TPF, and FDF, using a result for stopped stochastic processes. The central limit theorem (CLT) for the TPF is used to obtain an approximate expression for the $\lambda$-power, while the CLT for the FDF is used to introduce an approximate procedure for determining a suitably small nominal FDR that results in a speicified bound on the FDF with stipulated high probability. The paper also contains the results of a large simulation study covering a fairly substantial portion of the space of possible inputs encountered in application of the results in the design of a biomarker study, a micro-array experiment and a GWAS study.