On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA

TL;DR

This paper analyzes the properties of the sample covariance matrix for matrices with reduced effective rank, providing minimax rates, and evaluates the effectiveness of the scree plot method in PCA, with applications to functional data analysis.

Contribution

It offers a unified theoretical framework for covariance matrix estimation in reduced effective rank classes and assesses the finite-sample performance of the scree plot in PCA.

Findings

Sharp minimax rates for covariance estimation are derived.

The scree plot can be used for eigenvalue selection with proper thresholds.

Application demonstrated in functional principal component analysis.

Abstract

This work provides a unified analysis of the properties of the sample covariance matrix $\Sigma_n$ over the class of $p\times p$ population covariance matrices $\Sigma$ of reduced effective rank $r_e(\Sigma)$. This class includes scaled factor models and covariance matrices with decaying spectrum. We consider $r_e(\Sigma)$ as a measure of matrix complexity, and obtain sharp minimax rates on the operator and Frobenius norm of $\Sigma_n-\Sigma$, as a function of $r_e(\Sigma)$ and $\|\Sigma\|_2$, the operator norm of $\Sigma$. With guidelines offered by the optimal rates, we define classes of matrices of reduced effective rank over which $\Sigma_n$ is an accurate estimator. Within the framework of these classes, we perform a detailed finite sample theoretical analysis of the merits and limitations of the empirical scree plot procedure routinely used in PCA. We show that identifying jumps in the empirical spectrum that consistently estimate jumps in the spectrum of $\Sigma$ is not necessarily informative for other goals, for instance for the selection of those sample eigenvalues and eigenvectors that are consistent estimates of their population counterparts. The scree plot method can still be used for selecting consistent eigenvalues, for appropriate threshold levels. We provide a threshold construction and also give a rule for checking the consistency of the corresponding sample eigenvectors. We specialize these results and analysis to population covariance matrices with polynomially decaying spectra, and extend it to covariance operators with polynomially decaying spectra. An application to fPCA illustrates how our results can be used in functional data analysis.