CovarNav: Machine Unlearning via Model Inversion and Covariance Navigation

TL;DR

This paper introduces CovarNav, a novel machine unlearning method that uses model inversion and covariance navigation to effectively forget specific training data while preserving model performance.

Contribution

CovarNav is a new three-step approach combining model inversion, mislabeling, and gradient projection for efficient data removal in neural networks.

Findings

Effective forgetting of specific data points demonstrated on CIFAR-10 and Vggface2.

Outperforms recent benchmarks in machine unlearning tasks.

Maintains model accuracy while removing targeted training data.

Abstract

The rapid progress of AI, combined with its unprecedented public adoption and the propensity of large neural networks to memorize training data, has given rise to significant data privacy concerns. To address these concerns, machine unlearning has emerged as an essential technique to selectively remove the influence of specific training data points on trained models. In this paper, we approach the machine unlearning problem through the lens of continual learning. Given a trained model and a subset of training data designated to be forgotten (i.e., the "forget set"), we introduce a three-step process, named CovarNav, to facilitate this forgetting. Firstly, we derive a proxy for the model's training data using a model inversion attack. Secondly, we mislabel the forget set by selecting the most probable class that deviates from the actual ground truth. Lastly, we deploy a gradient projection method to minimize the cross-entropy loss on the modified forget set (i.e., learn incorrect labels for this set) while preventing forgetting of the inverted samples. We rigorously evaluate CovarNav on the CIFAR-10 and Vggface2 datasets, comparing our results with recent benchmarks in the field and demonstrating the efficacy of our proposed approach.