Instance-Level Difficulty: A Missing Perspective in Machine Unlearning

TL;DR

This paper investigates the challenges of unlearning individual data points in machine learning, revealing that difficulty depends on data and model factors rather than the unlearning method, highlighting a missing perspective in current research.

Contribution

It introduces an analysis of instance-level unlearning difficulty, identifying key factors affecting unlearning that are independent of specific algorithms.

Findings

Four factors influence unlearning difficulty.

Factors are independent of unlearning algorithms.

Difficulty depends on data and model, not method.

Abstract

Current research on deep machine unlearning primarily focuses on improving or evaluating the overall effectiveness of unlearning methods while overlooking the varying difficulty of unlearning individual training samples. As a result, the broader feasibility of machine unlearning remains under-explored. This paper studies the cruxes that make machine unlearning difficult through a thorough instance-level unlearning performance analysis over various unlearning algorithms and datasets. In particular, we summarize four factors that make unlearning a data point difficult, and we empirically show that these factors are independent of a specific unlearning algorithm but only relevant to the target model and its training data. Given these findings, we argue that machine unlearning research should pay attention to the instance-level difficulty of unlearning.