Surprising Asymptotic Conical Structure in Critical Sample Eigen-Directions

TL;DR

This paper uncovers a surprising asymptotic conical structure in the directions of sample eigenvectors in high-dimensional PCA models, revealing new insights into their behavior under various asymptotic regimes.

Contribution

It provides the first theoretical analysis of the asymptotic conical structure of sample eigenvectors in spike covariance models, including cases with indistinguishable eigenvalues.

Findings

Sample eigenvectors converge to a cone with a specific angle to population eigenvectors.

The angle between sample and population eigenvectors converges to a distribution in high-dimensional, low-sample scenarios.

The ratio of dimension to the product of sample size and spike size determines eigenvector consistency.

Abstract

The aim of this paper is to establish several deep theoretical properties of principal component analysis for multiple-component spike covariance models. Our new results reveal a surprising asymptotic conical structure in critical sample eigendirections under the spike models with distinguishable (or indistinguishable) eigenvalues, when the sample size and/or the number of variables (or dimension) tend to infinity. The consistency of the sample eigenvectors relative to their population counterparts is determined by the ratio between the dimension and the product of the sample size with the spike size. When this ratio converges to a nonzero constant, the sample eigenvector converges to a cone, with a certain angle to its corresponding population eigenvector.In the High Dimension, Low Sample Size case, the angle between the sample eigenvector and its population counterpart converges to a limiting distribution.Several generalizations of the multi-spike covariance models are also explored, and additional theoretical results are presented.