A New Basis for Sparse Principal Component Analysis

TL;DR

This paper introduces a novel sparse PCA method that uses orthogonal rotation to produce approximately sparse principal components, improving stability and variance explanation without requiring deflation or multiple tuning parameters.

Contribution

It proposes a new sparse PCA approach based on orthogonal rotation of principal components, differing from prior methods by not requiring the eigenbasis to be sparse and avoiding deflation.

Findings

More stable sparse components at the same sparsity level

Explains more variance compared to alternative methods

Effective in diverse applications like image coding and social network analysis

Abstract

Previous versions of sparse principal component analysis (PCA) have presumed that the eigen-basis (a $p \times k$ matrix) is approximately sparse. We propose a method that presumes the $p \times k$ matrix becomes approximately sparse after a $k \times k$ rotation. The simplest version of the algorithm initializes with the leading $k$ principal components. Then, the principal components are rotated with an $k \times k$ orthogonal rotation to make them approximately sparse. Finally, soft-thresholding is applied to the rotated principal components. This approach differs from prior approaches because it uses an orthogonal rotation to approximate a sparse basis. One consequence is that a sparse component need not to be a leading eigenvector, but rather a mixture of them. In this way, we propose a new (rotated) basis for sparse PCA. In addition, our approach avoids "deflation" and multiple tuning parameters required for that. Our sparse PCA framework is versatile; for example, it extends naturally to a two-way analysis of a data matrix for simultaneous dimensionality reduction of rows and columns. We provide evidence showing that for the same level of sparsity, the proposed sparse PCA method is more stable and can explain more variance compared to alternative methods. Through three applications -- sparse coding of images, analysis of transcriptome sequencing data, and large-scale clustering of social networks, we demonstrate the modern usefulness of sparse PCA in exploring multivariate data.