Federated Coordinate Descent for Privacy-Preserving Multiparty Linear Regression

TL;DR

This paper introduces Federated Coordinate Descent (FCD), a secure distributed algorithm for multiparty linear regression that effectively handles L1 regularization, ensuring privacy and high model performance.

Contribution

The paper proposes a novel secure multiparty coordinate descent scheme, FCD, capable of handling L1 regularization in distributed linear regression tasks.

Findings

FCD guarantees privacy through secure aggregation and perturbations.

FCD achieves comparable accuracy to centralized methods.

FCD is effective for linear, ridge, and lasso regressions.

Abstract

Distributed privacy-preserving regression schemes have been developed and extended in various fields, where multiparty collaboratively and privately run optimization algorithms, e.g., Gradient Descent, to learn a set of optimal parameters. However, traditional Gradient-Descent based methods fail to solve problems which contains objective functions with L1 regularization, such as Lasso regression. In this paper, we present Federated Coordinate Descent, a new distributed scheme called FCD, to address this issue securely under multiparty scenarios. Specifically, through secure aggregation and added perturbations, our scheme guarantees that: (1) no local information is leaked to other parties, and (2) global model parameters are not exposed to cloud servers. The added perturbations can eventually be eliminated by each party to derive a global model with high performance. We show that the FCD scheme fills the gap of multiparty secure Coordinate Descent methods and is applicable for general linear regressions, including linear, ridge and lasso regressions. Theoretical security analysis and experimental results demonstrate that FCD can be performed effectively and efficiently, and provide as low MAE measure as centralized methods under tasks of three types of linear regressions on real-world UCI datasets.