Seeking Consensus on Subspaces in Federated Principal Component Analysis

TL;DR

This paper introduces a novel federated PCA algorithm that enhances data privacy and reduces communication rounds by using a consensus-seeking approach based on subspace equalization, with proven convergence and empirical efficiency.

Contribution

It proposes a new ADMM-like algorithm for federated PCA that improves privacy and communication efficiency by relaxing feasibility constraints through subspace consensus.

Findings

Better data privacy protection than classic methods

Requires fewer communication rounds than existing algorithms

Proven convergence with complexity estimates

Abstract

In this paper, we develop an algorithm for federated principal component analysis (PCA) with emphases on both communication efficiency and data privacy. Generally speaking, federated PCA algorithms based on direct adaptations of classic iterative methods, such as simultaneous subspace iterations (SSI), are unable to preserve data privacy, while algorithms based on variable-splitting and consensus-seeking, such as alternating direction methods of multipliers (ADMM), lack in communication-efficiency. In this work, we propose a novel consensus-seeking formulation by equalizing subspaces spanned by splitting variables instead of equalizing variables themselves, thus greatly relaxing feasibility restrictions and allowing much faster convergence. Then we develop an ADMM-like algorithm with several special features to make it practically efficient, including a low-rank multiplier formula and techniques for treating subproblems. We establish that the proposed algorithm can better protect data privacy than classic methods adapted to the federated PCA setting. We derive convergence results, including a worst-case complexity estimate, for the proposed ADMM-like algorithm in the presence of the nonlinear equality constraints. Extensive empirical results are presented to show that the new algorithm, while enhancing data privacy, requires far fewer rounds of communication than existing peer algorithms for federated PCA.