Unification of Rare and Weak Detection Models using Moderate Deviations Analysis and Log-Chisquared P-values

TL;DR

This paper unifies rare and weak detection models by analyzing asymptotic performance of global tests using log-chisquared P-values, revealing optimal and sub-optimal testing strategies across various models.

Contribution

It provides a unified asymptotic framework for rare and weak models, characterizing test power and optimality using log-chisquared P-values across multiple hypothesis testing scenarios.

Findings

Berk-Jones and Higher Criticism tests are asymptotically optimal outside powerless regions.

Minimal P-value, FDR, and Fisher's tests are sub-optimal asymptotically.

The framework applies to Gaussian, Poisson, Binomial, and heteroscedastic models.

Abstract

Rare and Weak models for multiple hypothesis testing assume that only a small proportion of the tested hypotheses concern non-null effects and the individual effects are only moderately large, so they generally do not stand out individually, for example in a Bonferroni analysis. Such models have been studied in quite a few settings, for example in some cases studies focused on an underlying Gaussian means model for the hypotheses being tested; in others, Poisson and Binomial. Such seemingly different models have the following common structure. Summarizing the evidence of individual tests by the negative logarithm of its P-value, the model is asymptotically equivalent to a situation in which most negative log P-values have a standard exponential distribution but a small fraction might have an alternative distribution which is approximately noncentral chisquared on one degree of freedom. We characterize the asymptotic performance of global tests combining asymptotic log-chisquared P-values in terms of the chisquared mixture parameters: the scaling parameter controlling heteroscedasticity, the non-centrality parameter, and the parameter controlling the rarity of individual non-null effects. In a phase space involving the last two parameters, we derive a region where all tests are asymptotically powerless. Outside of this region, the Berk-Jones and the Higher Criticism tests have maximal power. Inference techniques based on the minimal P-value, false-discovery rate controlling, and Fisher's combination test have sub-optimal asymptotic phase diagrams. Our analysis yields the asymptotic power of global testing in various new rare and weak models, including two-sample heteroscedastic normal mixtures and binomial experiments with perturbed probabilities of success.