Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy

TL;DR

This paper reveals a vulnerability in machine unlearning systems where adversaries can significantly degrade model accuracy by submitting fake unlearning requests, highlighting the need for more robust verification methods.

Contribution

It introduces attack algorithms that target unlearning methods using data not in the training set and evaluates their effectiveness on image classification tasks.

Findings

White-box attacks reduce accuracy to 3.6% on CIFAR-10

Black-box attacks reduce accuracy to 8.5% on CIFAR-10

Most verification mechanisms fail to detect stealthy attacks

Abstract

Machine unlearning algorithms, designed for selective removal of training data from models, have emerged as a promising approach to growing privacy concerns. In this work, we expose a critical yet underexplored vulnerability in the deployment of unlearning systems: the assumption that the data requested for removal is always part of the original training set. We present a threat model where an attacker can degrade model accuracy by submitting adversarial unlearning requests for data not present in the training set. We propose white-box and black-box attack algorithms and evaluate them through a case study on image classification tasks using the CIFAR-10 and ImageNet datasets, targeting a family of widely used unlearning methods. Our results show extremely poor test accuracy following the attack: 3.6% on CIFAR-10 and 0.4% on ImageNet for white-box attacks, and 8.5% on CIFAR-10 and 1.3% on ImageNet for black-box attacks. Additionally, we evaluate various verification mechanisms to detect the legitimacy of unlearning requests and reveal the challenges in verification, as most of the mechanisms fail to detect stealthy attacks without severely impairing their ability to process valid requests. These findings underscore the urgent need for research on more robust request verification methods and unlearning protocols, should the deployment of machine unlearning systems become more prevalent in the future.