Manifold Quadratic Penalty Alternating Minimization for Sparse Principal Component Analysis

TL;DR

This paper introduces a simple, effective manifold quadratic penalty alternating minimization (MQPAM) method for sparse PCA, demonstrating competitive performance with existing algorithms in terms of sparsity and computational efficiency.

Contribution

The paper extends the quadratic penalty alternating minimization (QPAM) method to the manifold setting, creating a new, easy-to-implement algorithm for sparse PCA.

Findings

MQPAM performs comparably or better than existing algorithms.

MQPAM achieves good sparsity and efficiency in experiments.

The method is simple to implement with closed-form solutions.

Abstract

Optimization on the Stiefel manifold or with orthogonality constraints is an important problem in many signal processing and data analysis applications such as Sparse Principal Component Analysis (SPCA). Algorithms such as the Riemannian proximal gradient algorithms addressing this problem usually involve an intricate subproblem requiring a semi-smooth Newton method hence, simple and effective operator splitting methods extended to the manifold setting such as the Alternating Direction Method of Multipliers (ADMM) have been proposed. However, another simple operator-splitting method, the Quadratic Penalty Alternating Minimization (QPAM) method which has been successful in image processing to our knowledge, has not yet been extended to the manifold setting. In this paper, we propose a manifold QPAM (MQPAM) which is very simple to implement. The iterative scheme of the MQPAM consists of a Riemannian Gradient Descent (RGD) subproblem and a subproblem in the form of a proximal operator which has a closed-form solution. Experiments on the SPCA problem show that our proposed MQPAM is at par with or better than several other algorithms in terms of sparsity and CPU time.