On the usage of randomized p-values in the Schweder-Spjotvoll estimator

TL;DR

This paper proposes a new randomized p-value method to improve the estimation of the proportion of true null hypotheses in multiple testing, especially under composite null hypotheses where traditional p-values are biased.

Contribution

It introduces a tunable randomization approach for p-values that reduces bias and mean squared error in the Schweder-Spjotvoll estimator under composite null hypotheses.

Findings

Randomized p-values can minimize bias in $\

Numerical simulations show improved estimator performance with the new method.

Abstract

We are concerned with multiple test problems with composite null hypotheses and the estimation of the proportion $\pi_{0}$ of true null hypotheses. The Schweder-Spj\o tvoll estimator $\hat{\pi}_0$ utilizes marginal $p$-values and only works properly if the $p$-values that correspond to the true null hypotheses are uniformly distributed on $[0,1]$ ($\mathrm{Uni}[0,1]$-distributed). In the case of composite null hypotheses, marginal $p$-values are usually computed under least favorable parameter configurations (LFCs). Thus, they are stochastically larger than $\mathrm{Uni}[0,1]$ under non-LFCs in the null hypotheses. When using these LFC-based $p$-values, $\hat{\pi}_0$ tends to overestimate $\pi_{0}$. We introduce a new way of randomizing $p$-values that depends on a tuning parameter $c\in[0,1]$, such that $c=0$ and $c=1$ lead to $\mathrm{Uni}[0,1]$-distributed $p$-values, which are independent of the data, and to the original LFC-based $p$-values, respectively. For a certain value $c=c^{\star}$ the bias of $\hat{\pi}_0$ is minimized when using our randomized $p$-values. This often also entails a smaller mean squared error of the estimator as compared to the usage of the LFC-based $p$-values. We analyze these points theoretically, and we demonstrate them numerically in computer simulations under various standard statistical models.