Optimal Projected Variance Group-Sparse Block PCA

TL;DR

This paper introduces a new group sparse PCA formulation called GSMV that maximizes variance explained while promoting group sparsity, with an analytical solution and improved robustness over existing methods.

Contribution

It proposes a novel variance definition compatible with non-orthogonal components and a regularization approach leading to an efficient, convex optimization problem for group sparse PCA.

Findings

GSMV outperforms deflation in synthetic data for sparse structure retrieval.

GSMV is approximately three times faster than existing methods.

Application to real data demonstrates the effectiveness of group sparsity in variable selection.

Abstract

We address the problem of defining a group sparse formulation for Principal Components Analysis (PCA) - or its equivalent formulations as Low Rank approximation or Dictionary Learning problems - which achieves a compromise between maximizing the variance explained by the components and promoting sparsity of the loadings. So we propose first a new definition of the variance explained by non necessarily orthogonal components, which is optimal in some aspect and compatible with the principal components situation. Then we use a specific regularization of this variance by the group-$\ell_{1}$ norm to define a Group Sparse Maximum Variance (GSMV) formulation of PCA. The GSMV formulation achieves our objective by construction, and has the nice property that the inner non smooth optimization problem can be solved analytically, thus reducing GSMV to the maximization of a smooth and convex function under unit norm and orthogonality constraints, which generalizes Journee et al. (2010) to group sparsity. Numerical comparison with deflation on synthetic data shows that GSMV produces steadily slightly better and more robust results for the retrieval of hidden sparse structures, and is about three times faster on these examples. Application to real data shows the interest of group sparsity for variables selection in PCA of mixed data (categorical/numerical) .