Feature Grouping and Sparse Principal Component Analysis with Truncated Regularization

TL;DR

This paper introduces FGSPCA, a novel non-convex regularization-based PCA variant that simultaneously captures feature grouping and sparsity without prior knowledge, enhancing interpretability and reducing complexity.

Contribution

The paper proposes FGSPCA, a new PCA method using truncated regularization to automatically identify feature groups and sparse structures without prior information.

Findings

Demonstrates effective feature grouping and sparsity detection on synthetic data.

Shows improved interpretability and reduced model complexity.

Achieves promising results on real-world datasets.

Abstract

In this paper, we consider a new variant for principal component analysis (PCA), aiming to capture the grouping and/or sparse structures of factor loadings simultaneously. To achieve these goals, we employ a non-convex truncated regularization with naturally adjustable sparsity and grouping effects, and propose the Feature Grouping and Sparse Principal Component Analysis (FGSPCA). The proposed FGSPCA method encourages the factor loadings with similar values to collapse into disjoint homogeneous groups for feature grouping or into a special zero-valued group for feature selection, which in turn helps reducing model complexity and increasing model interpretation. Usually, existing structured PCA methods require prior knowledge to construct the regularization term. However, the proposed FGSPCA can simultaneously capture the grouping and/or sparse structures of factor loadings without any prior information. To solve the resulting non-convex optimization problem, we propose an alternating algorithm that incorporates the difference-of-convex programming, augmented Lagrange method and coordinate descent method. Experimental results demonstrate the promising performance and efficiency of the new method on both synthetic and real-world datasets. An R implementation of FGSPCA can be found on github {https://github.com/higeeks/FGSPCA}.