Rate-Privacy-Storage Tradeoff in Federated Learning with Top $r$ Sparsification

TL;DR

This paper explores the balance between communication efficiency, privacy, and storage in federated learning using top $r$ sparsification, proposing schemes that manage privacy and leakage trade-offs.

Contribution

It introduces schemes that achieve information-theoretic privacy in sparse federated learning and generalizes them to allow controlled information leakage.

Findings

Trade-off between communication, privacy, and storage costs established.

Schemes guarantee privacy of sparse updates with adjustable storage costs.

Framework for balancing information leakage and resource efficiency in FL.

Abstract

We investigate the trade-off between rate, privacy and storage in federated learning (FL) with top $r$ sparsification, where the users and the servers in the FL system only share the most significant $r$ and $r'$ fractions, respectively, of updates and parameters in the FL process, to reduce the communication cost. We present schemes that guarantee information theoretic privacy of the values and indices of the sparse updates sent by the users at the expense of a larger storage cost. To this end, we generalize the scheme to reduce the storage cost by allowing a certain amount of information leakage. Thus, we provide the general trade-off between the communication cost, storage cost, and information leakage in private FL with top $r$ sparsification, along the lines of two proposed schemes.