Dimension Reduction for Large-Scale Federated Data: Statistical Rate and Asymptotic Inference

TL;DR

This paper introduces FADI, a scalable PCA method for federated data that handles high dimensionality and large sample sizes efficiently, with theoretical guarantees and practical validation.

Contribution

The paper proposes FADI, a novel distributed PCA algorithm that accelerates computation for large-scale federated data while maintaining statistical accuracy and enabling inference.

Findings

FADI achieves the same error rate as traditional PCA under certain conditions.

FADI accelerates computation through parallel and distributed processing.

Theoretical analysis reveals a phase-transition in asymptotic distribution as parameters vary.

Abstract

In light of the rapidly growing large-scale data in federated ecosystems, the traditional principal component analysis (PCA) is often not applicable due to privacy protection considerations and large computational burden. Algorithms were proposed to lower the computational cost, but few can handle both high dimensionality and massive sample size under distributed settings. In this paper, we propose the FAst DIstributed (FADI) PCA method for federated data when both the dimension $d$ and the sample size $n$ are ultra-large, by simultaneously performing parallel computing along $d$ and distributed computing along $n$. Specifically, we utilize $L$ parallel copies of $p$-dimensional fast sketches to divide the computing burden along $d$ and aggregate the results distributively along the split samples. We present a general framework applicable to multiple statistical problems, and establish comprehensive theoretical results under the general framework. We show that FADI accelerates the computation while enjoying the same non-asymptotic error rate as the traditional PCA when $Lp \ge d$. We also derive inferential results that characterize the asymptotic distribution of FADI, and show a phase-transition phenomenon as $Lp$ increases. We perform extensive simulations to empirically validate our theoretical findings, and apply FADI to the 1000 Genomes data to study the population structure.