Enhancing Signal Proportion Estimation Through Leveraging Arbitrary Covariance Structures

TL;DR

This paper proposes a new signal proportion estimator that incorporates arbitrary covariance dependence, improving accuracy and robustness over traditional methods that assume independence.

Contribution

It introduces a novel estimator leveraging covariance dependence and extends confidence bounds with principal factor approximation for better performance.

Findings

Outperforms existing estimators in simulations.

Effectively detects weaker signals under dependence.

Provides theoretical insights into dependence and sparsity interactions.

Abstract

Accurately estimating the proportion of true signals among a large number of variables is crucial for enhancing the precision and reliability of scientific research. Traditional signal proportion estimators often assume independence among variables and specific signal sparsity conditions, limiting their applicability in real-world scenarios where such assumptions may not hold. This paper introduces a novel signal proportion estimator that leverages arbitrary covariance dependence information among variables, thereby improving performance across a wide range of sparsity levels and dependence structures. Building on previous work that provides lower confidence bounds for signal proportions, we extend this approach by incorporating the principal factor approximation procedure to account for variable dependence. Our theoretical insights offer a deeper understanding of how signal sparsity, signal intensity, and covariance dependence interact. By comparing the conditions for estimation consistency before and after dependence adjustment, we highlight the advantages of integrating dependence information across different contexts. This theoretical foundation not only validates the effectiveness of the new estimator but also guides its practical application, ensuring reliable use in diverse scenarios. Through extensive simulations, we demonstrate that our method outperforms state-of-the-art estimators in both estimation accuracy and the detection of weaker signals that might otherwise go undetected.