Innovated higher criticism for detecting sparse signals in correlated noise

TL;DR

This paper improves the higher criticism method for detecting sparse, weak signals in correlated noise by exploiting correlation structures, showing that correlation can enhance detection performance.

Contribution

It introduces a modified higher criticism approach that leverages correlation to improve detection of sparse signals, extending the method's effectiveness beyond independent noise scenarios.

Findings

Correlation can improve signal detection performance.

Optimal detection boundaries are characterized for correlated noise.

Performance is especially enhanced with polynomial decay or Toeplitz correlation matrices.

Abstract

Higher criticism is a method for detecting signals that are both sparse and weak. Although first proposed in cases where the noise variables are independent, higher criticism also has reasonable performance in settings where those variables are correlated. In this paper we show that, by exploiting the nature of the correlation, performance can be improved by using a modified approach which exploits the potential advantages that correlation has to offer. Indeed, it turns out that the case of independent noise is the most difficult of all, from a statistical viewpoint, and that more accurate signal detection (for a given level of signal sparsity and strength) can be obtained when correlation is present. We characterize the advantages of correlation by showing how to incorporate them into the definition of an optimal detection boundary. The boundary has particularly attractive properties when correlation decays at a polynomial rate or the correlation matrix is Toeplitz.