Optimal Strategies for Federated Learning Maintaining Client Privacy

TL;DR

This paper analyzes the tradeoff between privacy and performance in federated learning, proving that one local epoch per round is optimal and showing utility improves with more clients under differential privacy constraints.

Contribution

It provides a theoretical proof that training for one local epoch per global round optimizes performance while maintaining privacy, and explores how utility varies with the number of clients.

Findings

One local epoch per global round is optimal for privacy and performance.

Utility improves with more clients under differential privacy.

Experimental validation confirms theoretical insights.

Abstract

Federated Learning (FL) emerged as a learning method to enable the server to train models over data distributed among various clients. These clients are protective about their data being leaked to the server, any other client, or an external adversary, and hence, locally train the model and share it with the server rather than sharing the data. The introduction of sophisticated inferencing attacks enabled the leakage of information about data through access to model parameters. To tackle this challenge, privacy-preserving federated learning aims to achieve differential privacy through learning algorithms like DP-SGD. However, such methods involve adding noise to the model, data, or gradients, reducing the model's performance. This work provides a theoretical analysis of the tradeoff between model performance and communication complexity of the FL system. We formally prove that training for one local epoch per global round of training gives optimal performance while preserving the same privacy budget. We also investigate the change of utility (tied to privacy) of FL models with a change in the number of clients and observe that when clients are training using DP-SGD and argue that for the same privacy budget, the utility improved with increased clients. We validate our findings through experiments on real-world datasets. The results from this paper aim to improve the performance of privacy-preserving federated learning systems.