Support Recovery in Sparse PCA with Non-Random Missing Data

TL;DR

This paper presents a semidefinite relaxation algorithm for support recovery in sparse PCA with non-random missing data, providing theoretical guarantees and empirical validation under various structural conditions.

Contribution

The paper introduces a novel semidefinite relaxation approach for sparse PCA with incomplete data, with theoretical support for support recovery under general sampling schemes.

Findings

High-probability support recovery under certain spectral gap and noise conditions

Algorithm outperforms existing methods on synthetic and real data

Theoretical results extend to deterministic sampling schemes

Abstract

We analyze a practical algorithm for sparse PCA on incomplete and noisy data under a general non-random sampling scheme. The algorithm is based on a semidefinite relaxation of the $\ell_1$-regularized PCA problem. We provide theoretical justification that under certain conditions, we can recover the support of the sparse leading eigenvector with high probability by obtaining a unique solution. The conditions involve the spectral gap between the largest and second-largest eigenvalues of the true data matrix, the magnitude of the noise, and the structural properties of the observed entries. The concepts of algebraic connectivity and irregularity are used to describe the structural properties of the observed entries. We empirically justify our theorem with synthetic and real data analysis. We also show that our algorithm outperforms several other sparse PCA approaches especially when the observed entries have good structural properties. As a by-product of our analysis, we provide two theorems to handle a deterministic sampling scheme, which can be applied to other matrix-related problems.