A Decomposition Framework for Certifiably Optimal Orthogonal Sparse PCA

TL;DR

This paper introduces GS-SPCA, a novel algorithm for sparse PCA that guarantees sparsity, orthogonality, and optimality, and proposes acceleration strategies including branch-and-bound and a decomposition framework for multiple components.

Contribution

It presents a new SPCA algorithm with orthogonality and optimality guarantees and develops acceleration methods for computational efficiency, especially for multiple components.

Findings

GS-SPCA enforces sparsity, orthogonality, and optimality simultaneously.

Branch-and-bound enhances computational speed with controlled optimality.

Decomposition framework efficiently handles multiple principal components.

Abstract

Sparse Principal Component Analysis (SPCA) is an important technique for high-dimensional data analysis, improving interpretability by imposing sparsity on principal components. However, existing methods often fail to simultaneously guarantee sparsity, orthogonality, and optimality of the principal components. To address this challenge, this work introduces a novel Sparse Principal Component Analysis (SPCA) algorithm called \textsc{GS-SPCA} (SPCA with Gram-Schmidt Orthogonalization), which simultaneously enforces sparsity, orthogonality, and optimality. However, the original GS-SPCA algorithm is computationally expensive due to the inherent $\ell_0$-norm constraint. To address this issue, we propose two acceleration strategies: First, we combine \textbf{Branch-and-Bound} with the GS-SPCA algorithm. By incorporating this strategy, we are able to obtain $\varepsilon$-optimal solutions with a trade-off between precision and efficiency, significantly improving computational speed. Second, we propose a \textbf{decomposition framework} for efficiently solving \textbf{multiple} principal components. This framework approximates the covariance matrix using a block-diagonal matrix through a thresholding method, reducing the original SPCA problem to a set of block-wise subproblems on approximately block-diagonal matrices.