Debiased distributed PCA under high dimensional spiked model

TL;DR

This paper introduces a debiased distributed PCA method that corrects bias in high-dimensional settings, especially effective with few machines and sparse eigenvectors, backed by theoretical guarantees and empirical validation.

Contribution

It proposes a novel debiased distributed PCA algorithm with theoretical consistency under weaker conditions and adaptive sparsity detection, outperforming existing methods.

Findings

Achieves smaller estimation error with fewer machines.

Effectively handles sparse and non-sparse eigenvectors.

Outperforms existing distributed PCA methods in simulations and real data.

Abstract

We study distributed principal component analysis (PCA) in high-dimensional settings under the spiked model. In such regimes, sample eigenvectors can deviate significantly from population ones, introducing a persistent bias. Existing distributed PCA methods are sensitive to this bias, particularly when the number of machines is small. Their consistency typically relies on the number of machines tending to infinity. We propose a debiased distributed PCA algorithm that corrects the local bias before aggregation and incorporates a sparsity-detection step to adaptively handle sparse and non-sparse eigenvectors. Theoretically, we establish the consistency of our estimator under much weaker conditions compared to existing literature. In particular, our approach does not require symmetric innovations and only assumes a finite sixth moment. Furthermore, our method generally achieves smaller estimation error, especially when the number of machines is small. Empirically, extensive simulations and real data experiments demonstrate that our method consistently outperforms existing distributed PCA approaches. The advantage is especially prominent when the leading eigenvectors are sparse or the number of machines is limited. Our method and theoretical analysis are also applicable to the sample correlation matrix.