Coding for Straggler Mitigation in Federated Learning

TL;DR

This paper introduces a coded federated learning scheme that mitigates straggler effects, preserves privacy, and significantly speeds up training on image datasets without compromising accuracy.

Contribution

It proposes a novel scheme combining one-time padding and gradient codes for privacy-preserving, straggler-resilient federated learning with comparable training times to conventional methods.

Findings

Achieves 6.6x and 9.2x speed-up on MNIST and Fashion-MNIST datasets.

Maintains the same privacy level as conventional federated learning.

Provides effective straggler mitigation without sacrificing accuracy.

Abstract

We present a novel coded federated learning (FL) scheme for linear regression that mitigates the effect of straggling devices while retaining the privacy level of conventional FL. The proposed scheme combines one-time padding to preserve privacy and gradient codes to yield resiliency against stragglers and consists of two phases. In the first phase, the devices share a one-time padded version of their local data with a subset of other devices. In the second phase, the devices and the central server collaboratively and iteratively train a global linear model using gradient codes on the one-time padded local data. To apply one-time padding to real data, our scheme exploits a fixed-point arithmetic representation of the data. Unlike the coded FL scheme recently introduced by Prakash \emph{et al.}, the proposed scheme maintains the same level of privacy as conventional FL while achieving a similar training time. Compared to conventional FL, we show that the proposed scheme achieves a training speed-up factor of $6.6$ and $9.2$ on the MNIST and Fashion-MNIST datasets for an accuracy of $95\%$ and $85\%$, respectively.