Belt and Braces: When Federated Learning Meets Differential Privacy

TL;DR

This paper reviews the integration of federated learning and differential privacy, discussing current methods, challenges, and future directions for creating practical privacy-preserving machine learning systems.

Contribution

It provides a comprehensive review of FL with DP, categorizes existing paradigms, and discusses optimization principles for better utility-privacy tradeoffs.

Findings

Categorization of FL and DP paradigms

Analysis of tradeoffs between privacy and utility

Discussion of future research challenges

Abstract

Federated learning (FL) has great potential for large-scale machine learning (ML) without exposing raw data.Differential privacy (DP) is the de facto standard of privacy protection with provable guarantees.Advances in ML suggest that DP would be a perfect fit for FL with comprehensive privacy preservation. Hence, extensive efforts have been devoted to achieving practically usable FL with DP, which however is still challenging.Practitioners often not only are not fully aware of its development and categorization, but also face a hard choice between privacy and utility. Therefore, it calls for a holistic review of current advances and an investigation on the challenges and opportunities for highly usable FL systems with a DP guarantee. In this article, we first introduce the primary concepts of FL and DP, and highlight the benefits of integration. We then review the current developments by categorizing different paradigms and notions. Aiming at usable FL with DP, we present the optimization principles to seek a better tradeoff between model utility and privacy loss. Finally, we discuss future challenges in the emergent areas and relevant research topics.