Dynamic Principal Component Analysis in High Dimensions

TL;DR

This paper introduces a novel method for estimating dynamic eigenvectors in high-dimensional data, combining local smoothing and regularization, with proven theoretical properties and demonstrated effectiveness in simulations and real data.

Contribution

It proposes a unified framework for dynamic PCA in high dimensions, addressing the limitations of static eigenvector modeling in stochastic processes.

Findings

Effective estimation of dynamic eigenvectors demonstrated

Theoretical properties established under sparsity assumptions

Validated performance on simulated and real datasets

Abstract

Principal component analysis is a versatile tool to reduce dimensionality which has wide applications in statistics and machine learning. It is particularly useful for modeling data in high-dimensional scenarios where the number of variables $p$ is comparable to, or much larger than the sample size $n$. Despite an extensive literature on this topic, researchers have focused on modeling static principal eigenvectors, which are not suitable for stochastic processes that are dynamic in nature. To characterize the change in the entire course of high-dimensional data collection, we propose a unified framework to directly estimate dynamic eigenvectors of covariance matrices. Specifically, we formulate an optimization problem by combining the local linear smoothing and regularization penalty together with the orthogonality constraint, which can be effectively solved by manifold optimization algorithms. We show that our method is suitable for high-dimensional data observed under both common and irregular designs, and theoretical properties of the estimators are investigated under $l_q (0 \leq q \leq 1)$ sparsity. Extensive experiments demonstrate the effectiveness of the proposed method in both simulated and real data examples.