FedPower: Privacy-Preserving Distributed Eigenspace Estimation

TL;DR

FedPower introduces a federated learning algorithm for eigenspace estimation that enhances communication efficiency and privacy protection through local iterations, global aggregation with OPT, and differential privacy, with proven convergence bounds.

Contribution

The paper proposes FedPower, a novel federated eigenspace estimation algorithm combining local power iterations, OPT-based aggregation, and differential privacy, with theoretical convergence guarantees.

Findings

Effective eigenspace estimation in federated settings.

Improved communication efficiency over existing methods.

Strong privacy guarantees via differential privacy.

Abstract

Eigenspace estimation is fundamental in machine learning and statistics, which has found applications in PCA, dimension reduction, and clustering, among others. The modern machine learning community usually assumes that data come from and belong to different organizations. The low communication power and the possible privacy breaches of data make the computation of eigenspace challenging. To address these challenges, we propose a class of algorithms called \textsf{FedPower} within the federated learning (FL) framework. \textsf{FedPower} leverages the well-known power method by alternating multiple local power iterations and a global aggregation step, thus improving communication efficiency. In the aggregation, we propose to weight each local eigenvector matrix with {\it Orthogonal Procrustes Transformation} (OPT) for better alignment. To ensure strong privacy protection, we add Gaussian noise in each iteration by adopting the notion of \emph{differential privacy} (DP). We provide convergence bounds for \textsf{FedPower} that are composed of different interpretable terms corresponding to the effects of Gaussian noise, parallelization, and random sampling of local machines. Additionally, we conduct experiments to demonstrate the effectiveness of our proposed algorithms.