Graph Regularized PCA

TL;DR

Graph Regularized PCA (GR-PCA) enhances traditional PCA by incorporating data dependency structures via graph Laplacian regularization, leading to more interpretable components aligned with conditional relationships in high-dimensional, non-i.i.d. noise settings.

Contribution

This paper introduces GR-PCA, a novel graph-based regularization method for PCA that learns a sparse precision graph and biases loadings toward low-frequency Fourier modes, improving interpretability and structural fidelity.

Findings

GR-PCA concentrates variance on the intended support.

It produces loadings with lower graph-Laplacian energy.

It remains competitive in out-of-sample reconstruction.

Abstract

High-dimensional data often exhibit dependencies among variables that violate the isotropic-noise assumption under which principal component analysis (PCA) is optimal. For cases where the noise is not independent and identically distributed across features (i.e., the covariance is not spherical) we introduce Graph Regularized PCA (GR-PCA). It is a graph-based regularization of PCA that incorporates the dependency structure of the data features by learning a sparse precision graph and biasing loadings toward the low-frequency Fourier modes of the corresponding graph Laplacian. Consequently, high-frequency signals are suppressed, while graph-coherent low-frequency ones are preserved, yielding interpretable principal components aligned with conditional relationships. We evaluate GR-PCA on synthetic data spanning diverse graph topologies, signal-to-noise ratios, and sparsity levels. Compared to mainstream alternatives, it concentrates variance on the intended support, produces loadings with lower graph-Laplacian energy, and remains competitive in out-of-sample reconstruction. When high-frequency signals are present, the graph Laplacian penalty prevents overfitting, reducing the reconstruction accuracy but improving structural fidelity. The advantage over PCA is most pronounced when high-frequency signals are graph-correlated, whereas PCA remains competitive when such signals are nearly rotationally invariant. The procedure is simple to implement, modular with respect to the precision estimator, and scalable, providing a practical route to structure-aware dimensionality reduction that improves structural fidelity without sacrificing predictive performance.