Information Leakage from Data Updates in Machine Learning Models

TL;DR

This paper explores how adversaries can infer information about data updates in machine learning models by analyzing model snapshots, revealing vulnerabilities especially with rare attribute values and repeated updates.

Contribution

It introduces novel attack methods based on prediction confidence differences to infer data attribute changes during model updates, highlighting privacy risks.

Findings

Model snapshots leak more information than single models.

Rare attribute values are more vulnerable to inference attacks.

Repeated updates increase the likelihood of correct attribute inference.

Abstract

In this paper we consider the setting where machine learning models are retrained on updated datasets in order to incorporate the most up-to-date information or reflect distribution shifts. We investigate whether one can infer information about these updates in the training data (e.g., changes to attribute values of records). Here, the adversary has access to snapshots of the machine learning model before and after the change in the dataset occurs. Contrary to the existing literature, we assume that an attribute of a single or multiple training data points are changed rather than entire data records are removed or added. We propose attacks based on the difference in the prediction confidence of the original model and the updated model. We evaluate our attack methods on two public datasets along with multi-layer perceptron and logistic regression models. We validate that two snapshots of the model can result in higher information leakage in comparison to having access to only the updated model. Moreover, we observe that data records with rare values are more vulnerable to attacks, which points to the disparate vulnerability of privacy attacks in the update setting. When multiple records with the same original attribute value are updated to the same new value (i.e., repeated changes), the attacker is more likely to correctly guess the updated values since repeated changes leave a larger footprint on the trained model. These observations point to vulnerability of machine learning models to attribute inference attacks in the update setting.