Boundary Unlearning

TL;DR

Boundary Unlearning offers a fast and effective method to unlearn entire classes in deep neural networks by shifting decision boundaries, significantly reducing retraining time while maintaining utility and privacy.

Contribution

The paper introduces Boundary Unlearning, a novel approach focusing on decision boundary manipulation to efficiently unlearn classes in DNNs, outperforming traditional retraining methods.

Findings

Achieves 17x faster unlearning on CIFAR-10

Achieves 19x faster unlearning on Vggface2

Effectively forgets classes in image classification and face recognition

Abstract

The practical needs of the ``right to be forgotten'' and poisoned data removal call for efficient \textit{machine unlearning} techniques, which enable machine learning models to unlearn, or to forget a fraction of training data and its lineage. Recent studies on machine unlearning for deep neural networks (DNNs) attempt to destroy the influence of the forgetting data by scrubbing the model parameters. However, it is prohibitively expensive due to the large dimension of the parameter space. In this paper, we refocus our attention from the parameter space to the decision space of the DNN model, and propose Boundary Unlearning, a rapid yet effective way to unlearn an entire class from a trained DNN model. The key idea is to shift the decision boundary of the original DNN model to imitate the decision behavior of the model retrained from scratch. We develop two novel boundary shift methods, namely Boundary Shrink and Boundary Expanding, both of which can rapidly achieve the utility and privacy guarantees. We extensively evaluate Boundary Unlearning on CIFAR-10 and Vggface2 datasets, and the results show that Boundary Unlearning can effectively forget the forgetting class on image classification and face recognition tasks, with an expected speed-up of $17\times$ and $19\times$, respectively, compared with retraining from the scratch.