Evaluating Differentially Private Machine Learning in Practice

TL;DR

This paper investigates the practical privacy guarantees of differentially private machine learning, revealing a significant gap between theoretical bounds and actual privacy risks in common models like logistic regression and neural networks.

Contribution

It provides empirical analysis quantifying privacy-utility trade-offs and highlights the discrepancy between theoretical privacy guarantees and real-world attack effectiveness.

Findings

Large gap between theoretical privacy bounds and measured privacy loss.

Current mechanisms often fail to provide meaningful privacy or utility.

Trade-offs are severe in complex models, limiting practical use.

Abstract

Differential privacy is a strong notion for privacy that can be used to prove formal guarantees, in terms of a privacy budget, $\epsilon$, about how much information is leaked by a mechanism. However, implementations of privacy-preserving machine learning often select large values of $\epsilon$ in order to get acceptable utility of the model, with little understanding of the impact of such choices on meaningful privacy. Moreover, in scenarios where iterative learning procedures are used, differential privacy variants that offer tighter analyses are used which appear to reduce the needed privacy budget but present poorly understood trade-offs between privacy and utility. In this paper, we quantify the impact of these choices on privacy in experiments with logistic regression and neural network models. Our main finding is that there is a huge gap between the upper bounds on privacy loss that can be guaranteed, even with advanced mechanisms, and the effective privacy loss that can be measured using current inference attacks. Current mechanisms for differentially private machine learning rarely offer acceptable utility-privacy trade-offs with guarantees for complex learning tasks: settings that provide limited accuracy loss provide meaningless privacy guarantees, and settings that provide strong privacy guarantees result in useless models. Code for the experiments can be found here: https://github.com/bargavj/EvaluatingDPML