Releasing Malevolence from Benevolence: The Menace of Benign Data on Machine Unlearning

TL;DR

This paper introduces a novel attack called the Unlearning Usability Attack that exploits benign data to significantly degrade model performance and challenge existing unlearning techniques, highlighting new privacy and security concerns.

Contribution

The paper presents a model-agnostic, budget-friendly attack that uses benign data to undermine machine unlearning, revealing vulnerabilities in current defenses.

Findings

Unlearning benign data can reduce model accuracy by up to 50%.

Benign data can be used to challenge recent unlearning techniques.

Erasing synthetic benign data requires more resources than regular data.

Abstract

Machine learning models trained on vast amounts of real or synthetic data often achieve outstanding predictive performance across various domains. However, this utility comes with increasing concerns about privacy, as the training data may include sensitive information. To address these concerns, machine unlearning has been proposed to erase specific data samples from models. While some unlearning techniques efficiently remove data at low costs, recent research highlights vulnerabilities where malicious users could request unlearning on manipulated data to compromise the model. Despite these attacks' effectiveness, perturbed data differs from original training data, failing hash verification. Existing attacks on machine unlearning also suffer from practical limitations and require substantial additional knowledge and resources. To fill the gaps in current unlearning attacks, we introduce the Unlearning Usability Attack. This model-agnostic, unlearning-agnostic, and budget-friendly attack distills data distribution information into a small set of benign data. These data are identified as benign by automatic poisoning detection tools due to their positive impact on model training. While benign for machine learning, unlearning these data significantly degrades model information. Our evaluation demonstrates that unlearning this benign data, comprising no more than 1% of the total training data, can reduce model accuracy by up to 50%. Furthermore, our findings show that well-prepared benign data poses challenges for recent unlearning techniques, as erasing these synthetic instances demands higher resources than regular data. These insights underscore the need for future research to reconsider "data poisoning" in the context of machine unlearning.