Model Segmentation for Storage Efficient Private Federated Learning with Top $r$ Sparsification

TL;DR

This paper proposes two storage-efficient, privacy-preserving methods for federated learning with top $r$ sparsification, balancing storage costs and information leakage.

Contribution

It introduces novel schemes combining MDS coded storage and model segmentation to enhance privacy and reduce storage in federated learning.

Findings

Achieved privacy guarantees with reduced storage costs.

Demonstrated controllable information leakage trade-offs.

Provided practical schemes for private federated learning.

Abstract

In federated learning (FL) with top $r$ sparsification, millions of users collectively train a machine learning (ML) model locally, using their personal data by only communicating the most significant $r$ fraction of updates to reduce the communication cost. It has been shown that the values as well as the indices of these selected (sparse) updates leak information about the users' personal data. In this work, we investigate different methods to carry out user-database communications in FL with top $r$ sparsification efficiently, while guaranteeing information theoretic privacy of users' personal data. These methods incur considerable storage cost. As a solution, we present two schemes with different properties that use MDS coded storage along with a model segmentation mechanism to reduce the storage cost at the expense of a controllable amount of information leakage, to perform private FL with top $r$ sparsification.