Fundamental Limits of Membership Inference Attacks on Machine Learning Models

TL;DR

This paper explores the fundamental statistical limits of membership inference attacks on machine learning models, providing theoretical bounds and insights into factors affecting attack success.

Contribution

It offers the first theoretical analysis of the statistical limitations of MIAs, including bounds and conditions that influence attack effectiveness.

Findings

Discretizing data can improve model security against MIAs.

In overfitting non-linear regression, MIAs can have high success probability.

Data diversity bounds the effectiveness of membership inference attacks.

Abstract

Membership inference attacks (MIA) can reveal whether a particular data point was part of the training dataset, potentially exposing sensitive information about individuals. This article provides theoretical guarantees by exploring the fundamental statistical limitations associated with MIAs on machine learning models at large. More precisely, we first derive the statistical quantity that governs the effectiveness and success of such attacks. We then theoretically prove that in a non-linear regression setting with overfitting learning procedures, attacks may have a high probability of success. Finally, we investigate several situations for which we provide bounds on this quantity of interest. Interestingly, our findings indicate that discretizing the data might enhance the learning procedure's security. Specifically, it is demonstrated to be limited by a constant, which quantifies the diversity of the underlying data distribution. We illustrate those results through simple simulations.