Differential Privacy and Machine Learning: a Survey and Review

TL;DR

This survey reviews the integration of differential privacy into machine learning, discussing theoretical foundations, algorithms, and open challenges in balancing privacy with utility.

Contribution

It provides a comprehensive overview of privacy-preserving machine learning using differential privacy, including theoretical insights and open research questions.

Findings

Differential privacy effectively balances data utility and privacy in machine learning.

Theoretical bounds on loss functions for differentially private algorithms are established.

Open questions include leveraging public data and handling missing data in private datasets.

Abstract

The objective of machine learning is to extract useful information from data, while privacy is preserved by concealing information. Thus it seems hard to reconcile these competing interests. However, they frequently must be balanced when mining sensitive data. For example, medical research represents an important application where it is necessary both to extract useful information and protect patient privacy. One way to resolve the conflict is to extract general characteristics of whole populations without disclosing the private information of individuals. In this paper, we consider differential privacy, one of the most popular and powerful definitions of privacy. We explore the interplay between machine learning and differential privacy, namely privacy-preserving machine learning algorithms and learning-based data release mechanisms. We also describe some theoretical results that address what can be learned differentially privately and upper bounds of loss functions for differentially private algorithms. Finally, we present some open questions, including how to incorporate public data, how to deal with missing data in private datasets, and whether, as the number of observed samples grows arbitrarily large, differentially private machine learning algorithms can be achieved at no cost to utility as compared to corresponding non-differentially private algorithms.