How Worrying Are Privacy Attacks Against Machine Learning?

TL;DR

This paper reviews various privacy attacks on machine learning models, highlighting that many such attacks are less effective in real-world scenarios than previously believed, impacting privacy regulation considerations.

Contribution

It provides a comprehensive analysis of different privacy attack types against ML, challenging assumptions about their effectiveness in practical settings.

Findings

Most privacy attacks are less effective in real-world scenarios

Membership inference attacks have limited success outside controlled environments

Privacy risks may be overestimated based on prior literature

Abstract

In several jurisdictions, the regulatory framework on the release and sharing of personal data is being extended to machine learning (ML). The implicit assumption is that disclosing a trained ML model entails a privacy risk for any personal data used in training comparable to directly releasing those data. However, given a trained model, it is necessary to mount a privacy attack to make inferences on the training data. In this concept paper, we examine the main families of privacy attacks against predictive and generative ML, including membership inference attacks (MIAs), property inference attacks, and reconstruction attacks. Our discussion shows that most of these attacks seem less effective in the real world than what a prima face interpretation of the related literature could suggest.