Practical Lossless Federated Singular Vector Decomposition over Billion-Scale Data

TL;DR

FedSVD introduces a lossless federated SVD method that ensures high accuracy and efficiency over billion-scale data, outperforming existing privacy-preserving solutions in speed and precision.

Contribution

The paper presents FedSVD, a novel lossless federated SVD approach that removes privacy-preserving noise and avoids data inflation, achieving superior accuracy and efficiency.

Findings

FedSVD is over 10,000 times faster than HE-based methods.

FedSVD has 10 orders of magnitude smaller error than DP-based solutions.

FedSVD outperforms state-of-the-art solutions in federated PCA, LR, and LSA tasks.

Abstract

With the enactment of privacy-preserving regulations, e.g., GDPR, federated SVD is proposed to enable SVD-based applications over different data sources without revealing the original data. However, many SVD-based applications cannot be well supported by existing federated SVD solutions. The crux is that these solutions, adopting either differential privacy (DP) or homomorphic encryption (HE), suffer from accuracy loss caused by unremovable noise or degraded efficiency due to inflated data. In this paper, we propose FedSVD, a practical lossless federated SVD method over billion-scale data, which can simultaneously achieve lossless accuracy and high efficiency. At the heart of FedSVD is a lossless matrix masking scheme delicately designed for SVD: 1) While adopting the masks to protect private data, FedSVD completely removes them from the final results of SVD to achieve lossless accuracy; and 2) As the masks do not inflate the data, FedSVD avoids extra computation and communication overhead during the factorization to maintain high efficiency. Experiments with real-world datasets show that FedSVD is over 10000 times faster than the HE-based method and has 10 orders of magnitude smaller error than the DP-based solution on SVD tasks. We further build and evaluate FedSVD over three real-world applications: principal components analysis (PCA), linear regression (LR), and latent semantic analysis (LSA), to show its superior performance in practice. On federated LR tasks, compared with two state-of-the-art solutions: FATE and SecureML, FedSVD-LR is 100 times faster than SecureML and 10 times faster than FATE.