Federated Learning with Bayesian Differential Privacy

TL;DR

This paper introduces Bayesian differential privacy to federated learning, providing tighter privacy guarantees and improved model accuracy by reducing noise and communication rounds, especially in sensitive applications like medical image classification.

Contribution

It adapts Bayesian differential privacy to federated learning, offering sharper privacy bounds and efficiency improvements over existing differential privacy methods.

Findings

Achieves privacy budget below 1 at client level and below 0.1 at instance level.

Reduces noise, improving model accuracy.

Decreases number of communication rounds.

Abstract

We consider the problem of reinforcing federated learning with formal privacy guarantees. We propose to employ Bayesian differential privacy, a relaxation of differential privacy for similarly distributed data, to provide sharper privacy loss bounds. We adapt the Bayesian privacy accounting method to the federated setting and suggest multiple improvements for more efficient privacy budgeting at different levels. Our experiments show significant advantage over the state-of-the-art differential privacy bounds for federated learning on image classification tasks, including a medical application, bringing the privacy budget below 1 at the client level, and below 0.1 at the instance level. Lower amounts of noise also benefit the model accuracy and reduce the number of communication rounds.