Fast Sparse PCA via Positive Semidefinite Projection for Unsupervised Feature Selection

TL;DR

This paper introduces a fast convex sparse PCA method with a positive semidefinite projection constraint for improved unsupervised feature selection, demonstrating superior efficiency and effectiveness on real-world data.

Contribution

It proves the optimal solutions of convex SPCA models lie on the PSD cone and proposes a two-step PSD projection algorithm for faster convergence.

Findings

The proposed method outperforms existing SPCA techniques in feature selection accuracy.

The PSD projection accelerates convergence of convex SPCA models.

Experiments confirm the effectiveness and efficiency of the proposed approach.

Abstract

In the field of unsupervised feature selection, sparse principal component analysis (SPCA) methods have attracted more and more attention recently. Compared to spectral-based methods, SPCA methods don't rely on the construction of a similarity matrix and show better feature selection ability on real-world data. The original SPCA formulates a nonconvex optimization problem. Existing convex SPCA methods reformulate SPCA as a convex model by regarding the reconstruction matrix as an optimization variable. However, they are lack of constraints equivalent to the orthogonality restriction in SPCA, leading to larger solution space. In this paper, it's proved that the optimal solution to a convex SPCA model falls onto the Positive Semidefinite (PSD) cone. A standard convex SPCA-based model with PSD constraint for unsupervised feature selection is proposed. Further, a two-step fast optimization algorithm via PSD projection is presented to solve the proposed model. Two other existing convex SPCA-based models are also proven to have their solutions optimized on the PSD cone in this paper. Therefore, the PSD versions of these two models are proposed to accelerate their convergence as well. We also provide a regularization parameter setting strategy for our proposed method. Experiments on synthetic and real-world datasets demonstrate the effectiveness and efficiency of the proposed methods.