Statistical Limits of Sparse Mixture Detection

TL;DR

This paper characterizes the fundamental limits of detecting sparse mixtures, generalizes previous univariate results, and establishes conditions for the optimality of certain testing procedures using large deviations theory.

Contribution

It provides a comprehensive phase transition analysis and adaptive optimality conditions for sparse mixture detection, extending prior work to more general settings.

Findings

Explicit phase transition characterization for sparse mixture detection

Sufficient condition for adaptive optimality of Higher Criticism tests

Unified large deviations perspective on detection limits

Abstract

We consider the problem of detecting a general sparse mixture and obtain an explicit characterization of the phase transition under some conditions, generalizing the univariate results of Cai and Wu. Additionally, we provide a sufficient condition for the adaptive optimality of a Higher Criticism type testing statistic formulated by Gao and Ma. In the course of establishing these results, we offer a unified perspective through the large deviations theory. The phase transition and adaptive optimality we establish are direct consequences of the large deviation principle of the normalized log-likelihood ratios between the null and the signal distributions.