DP-REC: Private & Communication-Efficient Federated Learning

TL;DR

DP-REC is a novel federated learning method that combines highly compressed communication via Relative Entropy Coding with differential privacy, significantly reducing communication costs while maintaining strong privacy guarantees.

Contribution

We introduce DP-REC, a new approach that unifies compression and differential privacy in federated learning using a modified REC technique.

Findings

Drastically reduces communication costs in federated learning.

Provides privacy guarantees comparable to state-of-the-art methods.

Demonstrates effectiveness through experimental validation.

Abstract

Privacy and communication efficiency are important challenges in federated training of neural networks, and combining them is still an open problem. In this work, we develop a method that unifies highly compressed communication and differential privacy (DP). We introduce a compression technique based on Relative Entropy Coding (REC) to the federated setting. With a minor modification to REC, we obtain a provably differentially private learning algorithm, DP-REC, and show how to compute its privacy guarantees. Our experiments demonstrate that DP-REC drastically reduces communication costs while providing privacy guarantees comparable to the state-of-the-art.