Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design

TL;DR

This paper introduces the Interpolated MVU mechanism, a novel privacy-aware compression method for federated learning that improves scalability and privacy-utility trade-offs, achieving state-of-the-art communication efficiency.

Contribution

It extends the MVU mechanism with an interpolation procedure, enhancing privacy analysis and scalability in private federated learning.

Findings

Achieves state-of-the-art results on multiple datasets.

Offers a more scalable privacy analysis method.

Improves the privacy-utility trade-off in federated learning.

Abstract

In private federated learning (FL), a server aggregates differentially private updates from a large number of clients in order to train a machine learning model. The main challenge in this setting is balancing privacy with both classification accuracy of the learnt model as well as the number of bits communicated between the clients and server. Prior work has achieved a good trade-off by designing a privacy-aware compression mechanism, called the minimum variance unbiased (MVU) mechanism, that numerically solves an optimization problem to determine the parameters of the mechanism. This paper builds upon it by introducing a new interpolation procedure in the numerical design process that allows for a far more efficient privacy analysis. The result is the new Interpolated MVU mechanism that is more scalable, has a better privacy-utility trade-off, and provides SOTA results on communication-efficient private FL on a variety of datasets.