Private Federated Submodel Learning with Sparsification

TL;DR

This paper presents a novel private federated submodel learning scheme that incorporates sparsification, enabling efficient and privacy-preserving updates of relevant submodels with reduced communication costs.

Contribution

It introduces a new scheme for private read and write operations in federated submodel learning that handles sparsification without revealing sensitive information.

Findings

Achieves lower reading and writing costs with sparsification.

Ensures privacy of submodel index and update coordinates.

Supports arbitrary submodel parameter updates.

Abstract

We investigate the problem of private read update write (PRUW) in federated submodel learning (FSL) with sparsification. In FSL, a machine learning model is divided into multiple submodels, where each user updates only the submodel that is relevant to the user's local data. PRUW is the process of privately performing FSL by reading from and writing to the required submodel without revealing the submodel index, or the values of updates to the databases. Sparsification is a widely used concept in learning, where the users update only a small fraction of parameters to reduce the communication cost. Revealing the coordinates of these selected (sparse) updates leaks privacy of the user. We show how PRUW in FSL can be performed with sparsification. We propose a novel scheme which privately reads from and writes to arbitrary parameters of any given submodel, without revealing the submodel index, values of updates, or the coordinates of the sparse updates, to databases. The proposed scheme achieves significantly lower reading and writing costs compared to what is achieved without sparsification.