A universal test on spikes in a high-dimensional generalized spiked model and its applications

TL;DR

This paper introduces a universal test for detecting spikes in high-dimensional covariance matrices, applicable without Gaussian assumptions, and demonstrates improved noise variance estimation and robustness in simulations.

Contribution

It develops a general test statistic for spikes in high-dimensional models without requiring Gaussianity or diagonal covariance structures, extending applicability.

Findings

Test statistic follows a central limit theorem derived via random matrix theory.

Proposed method accurately sizes tests and is robust to non-Gaussian populations.

Estimator of noise variance outperforms existing methods in accuracy.

Abstract

This paper aims to test the number of spikes in a generalized spiked covariance matrix, the spiked eigenvalues of which may be extremely larger or smaller than the non-spiked ones. For a high-dimensional problem, we first propose a general test statistic and derive its central limit theorem by random matrix theory without a Gaussian population constraint. We then apply the result to estimate the noise variance and test the equality of the smallest roots in generalized spiked models. Simulation studies showed that the proposed test method was correctly sized, and the power outcomes showed the robustness of our statistic to deviations from a Gaussian population. Moreover, our estimator of the noise variance resulted in much smaller mean absolute errors and mean squared errors than existing methods. In contrast to previously developed methods, we eliminated the strict conditions of diagonal or block-wise diagonal form of the population covariance matrix and extend the work to a wider range without the assumption of normality. Thus, the proposed method is more suitable for real problems.