Stochastic Coded Federated Learning with Convergence and Privacy Guarantees

TL;DR

This paper introduces a stochastic coded federated learning framework that enhances convergence speed and privacy guarantees by using coded datasets and differential privacy analysis, addressing straggler issues in heterogeneous FL environments.

Contribution

The paper proposes a novel coded FL framework with privacy-preserving data encoding, convergence analysis, and a privacy-performance tradeoff, improving upon traditional federated learning methods.

Findings

SCFL achieves faster convergence compared to standard FL.

The framework provides quantifiable privacy guarantees via MI-DP.

Numerical results validate the effectiveness of SCFL in privacy and convergence.

Abstract

Federated learning (FL) has attracted much attention as a privacy-preserving distributed machine learning framework, where many clients collaboratively train a machine learning model by exchanging model updates with a parameter server instead of sharing their raw data. Nevertheless, FL training suffers from slow convergence and unstable performance due to stragglers caused by the heterogeneous computational resources of clients and fluctuating communication rates. This paper proposes a coded FL framework to mitigate the straggler issue, namely stochastic coded federated learning (SCFL). In this framework, each client generates a privacy-preserving coded dataset by adding additive noise to the random linear combination of its local data. The server collects the coded datasets from all the clients to construct a composite dataset, which helps to compensate for the straggling effect. In the training process, the server as well as clients perform mini-batch stochastic gradient descent (SGD), and the server adds a make-up term in model aggregation to obtain unbiased gradient estimates. We characterize the privacy guarantee by the mutual information differential privacy (MI-DP) and analyze the convergence performance in federated learning. Besides, we demonstrate a privacy-performance tradeoff of the proposed SCFL method by analyzing the influence of the privacy constraint on the convergence rate. Finally, numerical experiments corroborate our analysis and show the benefits of SCFL in achieving fast convergence while preserving data privacy.