Improved Localized Machine Unlearning Through the Lens of Memorization

TL;DR

This paper introduces a novel localized unlearning method called DEL, which effectively removes data influence with minimal parameter modification, improving unlearning performance and test accuracy.

Contribution

It proposes an improved localization strategy inspired by memorization literature and a new unlearning algorithm, DEL, that outperforms existing methods in efficiency and accuracy.

Findings

DEL achieves state-of-the-art unlearning metrics.

DEL modifies fewer parameters while maintaining high performance.

DEL outperforms existing localized unlearning methods in accuracy.

Abstract

Machine unlearning refers to removing the influence of a specified subset of training data from a machine learning model, efficiently, after it has already been trained. This is important for key applications, including making the model more accurate by removing outdated, mislabeled, or poisoned data. In this work, we study localized unlearning, where the unlearning algorithm operates on a (small) identified subset of parameters. Drawing inspiration from the memorization literature, we propose an improved localization strategy that yields strong results when paired with existing unlearning algorithms. We also propose a new unlearning algorithm, Deletion by Example Localization (DEL), that resets the parameters deemed-to-be most critical according to our localization strategy, and then finetunes them. Our extensive experiments on different datasets, forget sets and metrics reveal that DEL sets a new state-of-the-art for unlearning metrics, against both localized and full-parameter methods, while modifying a small subset of parameters, and outperforms the state-of-the-art localized unlearning in terms of test accuracy too.