A General Framework For Consistency of Principal Component Analysis

TL;DR

This paper introduces a comprehensive asymptotic framework for analyzing the consistency of PCA, revealing how dimension, sample size, and eigenvalue spikes influence its performance across various scenarios.

Contribution

It develops a unified framework that encompasses existing asymptotic regimes and explores new scenarios, providing insights into PCA's consistency and convergence rates under general models.

Findings

Dimension hampers PCA consistency

Sample size and spike strength promote PCA consistency

The framework clarifies relationships among key factors affecting PCA

Abstract

A general asymptotic framework is developed for studying consis- tency properties of principal component analysis (PCA). Our frame- work includes several previously studied domains of asymptotics as special cases and allows one to investigate interesting connections and transitions among the various domains. More importantly, it enables us to investigate asymptotic scenarios that have not been considered before, and gain new insights into the consistency, subspace consistency and strong inconsistency regions of PCA and the boundaries among them. We also establish the corresponding convergence rate within each region. Under general spike covariance models, the dimension (or the number of variables) discourages the consistency of PCA, while the sample size and spike information (the relative size of the population eigenvalues) encourages PCA consistency. Our framework nicely illustrates the relationship among these three types of information in terms of dimension, sample size and spike size, and rigorously characterizes how their relationships affect PCA consistency.