An Interpretable and Stable Framework for Sparse Principal Component Analysis

TL;DR

This paper introduces SP-SPCA, a novel adaptive regularization method for sparse PCA that improves interpretability, stability, and feature selection in high-dimensional noisy data.

Contribution

It proposes a new equilibrium parameter in the regularization framework, enhancing sparse PCA's adaptability and performance in complex data settings.

Findings

Outperforms standard sparse PCA in simulations

Effectively filters noise variables

Reduces model complexity while maintaining explanatory power

Abstract

Sparse principal component analysis (SPCA) addresses the poor interpretability and variable redundancy often encountered by principal component analysis (PCA) in high-dimensional data. However, SPCA typically imposes uniform penalties on variables and does not account for differences in variable importance, which may lead to unstable performance in highly noisy or structurally complex settings. We propose SP-SPCA, a method that introduces a single equilibrium parameter into the regularization framework to adaptively adjust variable penalties. This modification of the L2 penalty provides flexible control over the trade-off between sparsity and explained variance while maintaining computational efficiency. Simulation studies show that the proposed method consistently outperforms standard sparse principal component methods in identifying sparse loading patterns, filtering noise variables, and preserving cumulative variance, especially in high-dimensional and noisy settings. Empirical applications to crime and financial market data further demonstrate its practical utility. In real data analyses, the method selects fewer but more relevant variables, thereby reducing model complexity while maintaining explanatory power. Overall, the proposed approach offers a robust and efficient alternative for sparse modeling in complex high-dimensional data, with clear advantages in stability, feature selection, and interpretability