FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysis

TL;DR

FAST-PCA is a novel distributed algorithm that efficiently and exactly computes principal components with linear convergence, suitable for large-scale, distributed datasets, improving over existing methods in communication and accuracy.

Contribution

This paper introduces FAST-PCA, the first distributed PCA algorithm that is both communication-efficient and guarantees exact, linearly convergent principal component computation.

Findings

Converges linearly to the true principal components.

Reduces communication overhead compared to existing methods.

Achieves exact and global convergence in distributed PCA.

Abstract

Principal Component Analysis (PCA) is a fundamental data preprocessing tool in the world of machine learning. While PCA is often thought of as a dimensionality reduction method, the purpose of PCA is actually two-fold: dimension reduction and uncorrelated feature learning. Furthermore, the enormity of the dimensions and sample size in the modern day datasets have rendered the centralized PCA solutions unusable. In that vein, this paper reconsiders the problem of PCA when data samples are distributed across nodes in an arbitrarily connected network. While a few solutions for distributed PCA exist, those either overlook the uncorrelated feature learning aspect of the PCA, tend to have high communication overhead that makes them inefficient and/or lack `exact' or `global' convergence guarantees. To overcome these aforementioned issues, this paper proposes a distributed PCA algorithm termed FAST-PCA (Fast and exAct diSTributed PCA). The proposed algorithm is efficient in terms of communication and is proven to converge linearly and exactly to the principal components, leading to dimension reduction as well as uncorrelated features. The claims are further supported by experimental results.