Sparse Principal Component Analysis with Energy Profile Dependent Sample Complexity

TL;DR

This paper introduces SEP, an iterative method for sparse PCA that adapts to uneven energy distribution in the signal, achieving improved sample complexity bounds and demonstrating superior empirical performance.

Contribution

We propose SEP, a novel spectral method for sparse PCA that adapts to energy profiles, with theoretical guarantees and empirical validation.

Findings

SEP achieves sample complexity of order  p s^2(p)  n log n.

SEP outperforms existing algorithms in simulations across various energy profiles.

A single power iteration post-process refines the estimator to attain uniform statistical error.

Abstract

We study sparse principal component analysis in the high-dimensional, sample-limited regime, aiming to recover a leading component supported on a few coordinates. Despite extensive progress, most methods and analyses are tailored to the flat-spike case, offering little guidance when spike energy is unevenly distributed across the support. Motivated by this, we propose Spectral Energy Pursuit (SEP), an effective iterative scheme that repeatedly screens and reselects coordinates, with a sample complexity that adapts to the energy profile. We develop our framework around a structure function \(s(p)\) that quantifies how spike energy accumulates over its top \(p\) entries. We establish that SEP succeeds with a sample size of order \(\max_{1\le p\le k} p\,s^2(p)\,\log n\), which matches the classical \(k^2\log n\) sample complexity for flat spikes and improves toward the \(k\log n\) regime as the profile becomes more concentrated. As a lightweight post-processing, a single truncated power iteration is proven to enable the final estimator to attain a uniform statistical error bound. Empirical simulations across flat, power-law, and exponential signals validate that SEP adapts to profile structure without tuning and outperforms existing algorithms.