Data Duplication: A Novel Multi-Purpose Attack Paradigm in Machine Unlearning

TL;DR

This paper investigates how data duplication affects machine unlearning, revealing vulnerabilities where duplicates can persist post-unlearning, degrade model performance, and evade detection across various paradigms.

Contribution

It introduces novel adversarial duplication techniques and analyzes their impact on unlearning effectiveness and detection methods across multiple unlearning paradigms.

Findings

Retraining from scratch may fail to unlearn duplicated data effectively.

Duplicated data can cause significant model degradation.

Crafted duplicates can evade de-duplication detection.

Abstract

Duplication is a prevalent issue within datasets. Existing research has demonstrated that the presence of duplicated data in training datasets can significantly influence both model performance and data privacy. However, the impact of data duplication on the unlearning process remains largely unexplored. This paper addresses this gap by pioneering a comprehensive investigation into the role of data duplication, not only in standard machine unlearning but also in federated and reinforcement unlearning paradigms. Specifically, we propose an adversary who duplicates a subset of the target model's training set and incorporates it into the training set. After training, the adversary requests the model owner to unlearn this duplicated subset, and analyzes the impact on the unlearned model. For example, the adversary can challenge the model owner by revealing that, despite efforts to unlearn it, the influence of the duplicated subset remains in the model. Moreover, to circumvent detection by de-duplication techniques, we propose three novel near-duplication methods for the adversary, each tailored to a specific unlearning paradigm. We then examine their impacts on the unlearning process when de-duplication techniques are applied. Our findings reveal several crucial insights: 1) the gold standard unlearning method, retraining from scratch, fails to effectively conduct unlearning under certain conditions; 2) unlearning duplicated data can lead to significant model degradation in specific scenarios; and 3) meticulously crafted duplicates can evade detection by de-duplication methods.