Weak Signal Inclusion Under Sparsity and Dependence

TL;DR

This paper introduces a new method for detecting weak signals in high-dimensional data, effectively controlling false negatives even under complex dependence structures, with applications in biomedical data analysis.

Contribution

The paper develops a novel false negative control method that accounts for arbitrary covariance dependence, improving detection of weak signals in high-dimensional sparse inference.

Findings

Outperforms existing methods in false negative control.

Effectively retains high proportion of signals under dependence.

Demonstrates practical utility in fMRI data analysis.

Abstract

We consider the scenario where important signals are not strong enough to be separable from a large amount of noise. Such weak signals commonly exist in large-scale data analysis and play vital roles in many biomedical applications. Existing methods however are mostly underpowered for such weak signals. We address the challenge from the perspective of false negative control and develop a new method to efficiently regulate false negative proportion at a user-specified level. The new method is developed in a realistic setting with arbitrary covariance dependence between variables. We calibrate the overall dependence through a parameter whose scale is compatible with the existing phase diagram in high-dimensional sparse inference. Utilizing the new calibration, we asymptotically explicate the joint effect of covariance dependence, signal sparsity, and signal intensity on the proposed method. We interpret the results using a new phase diagram, which shows that the proposed method can efficiently retain a high proportion of signals even when they cannot be well-separated from noise. Finite sample performance of the proposed method is compared to those of several existing methods in simulation studies. The proposed method outperforms the others in adapting to a user-specified false negative control level. We apply the new method to analyze an fMRI dataset to locate voxels that are functionally relevant to saccadic eye movements. The new method exhibits a nice balance in identifying functional relevant regions and avoiding excessive noise voxels.