Evaluating Membership Inference Attacks in heterogeneous-data setups

TL;DR

This paper investigates how membership inference attacks perform under heterogeneous data conditions, introducing a new metric for data distribution differences and comparing simulation methodologies to better reflect real-world scenarios.

Contribution

It introduces a novel heterogeneity metric for tabular data distributions and compares two methods for simulating data heterogeneity in MIA evaluations.

Findings

MIA accuracy varies significantly with data heterogeneity setup.

Current evaluation methods lack standardization for heterogeneous data.

Heterogeneity impacts the reliability of privacy risk assessments.

Abstract

Among all privacy attacks against Machine Learning (ML), membership inference attacks (MIA) attracted the most attention. In these attacks, the attacker is given an ML model and a data point, and they must infer whether the data point was used for training. The attacker also has an auxiliary dataset to tune their inference algorithm. Attack papers commonly simulate setups in which the attacker's and the target's datasets are sampled from the same distribution. This setting is convenient to perform experiments, but it rarely holds in practice. ML literature commonly starts with similar simplifying assumptions (i.e., "i.i.d." datasets), and later generalizes the results to support heterogeneous data distributions. Similarly, our work makes a first step in the generalization of the MIA evaluation to heterogeneous data. First, we design a metric to measure the heterogeneity between any pair of tabular data distributions. This metric provides a continuous scale to analyze the phenomenon. Second, we compare two methodologies to simulate a data heterogeneity between the target and the attacker. These setups provide opposite performances: 90% attack accuracy vs. 50% (i.e., random guessing). Our results show that the MIA accuracy depends on the experimental setup; and even if research on MIA considers heterogeneous data setups, we have no standardized baseline of how to simulate it. The lack of such a baseline for MIA experiments poses a significant challenge to risk assessments in real-world machine learning scenarios.