Sparse principal component analysis and its $l_1$-relaxation

TL;DR

This paper establishes a formal relationship between sparse PCA with l0 constraints and its l1-relaxation, showing their optimal values are within a constant factor, which aids interpretability in data analysis.

Contribution

It proves, for the first time, that the optimal values of sparse PCA with l0 and l1 constraints are within a constant factor, linking the two approaches.

Findings

Optimal objective values of l0 and l1 sparse PCA are within a constant factor.

First formal relationship established between l0 and l1 constrained sparse PCA.

Results are independent of the data.

Abstract

Principal component analysis (PCA) is one of the most widely used dimensionality reduction methods in scientific data analysis. In many applications, for additional interpretability, it is desirable for the factor loadings to be sparse, that is, we solve PCA with an additional cardinality (l0) constraint. The resulting optimization problem is called the sparse principal component analysis (SPCA). One popular approach to achieve sparsity is to replace the l0 constraint by an l1 constraint. In this paper, we prove that, independent of the data, the optimal objective function value of the problem with l0 constraint is within a constant factor of the the optimal objective function value of the problem with l1 constraint. To the best of our knowledge, this is the first formal relationship established between the l0 and the l1 constraint version of the problem.