Towards Aligned Data Removal via Twin Machine Unlearning

TL;DR

This paper introduces Twin Machine Unlearning (TMU), a novel method that improves data removal in machine learning models by aligning unlearned models with the original, maintaining accuracy while ensuring data privacy compliance.

Contribution

The paper proposes TMU, a new approach that aligns unlearned models with the original, addressing limitations of previous methods that reduce accuracy after data removal.

Findings

TMU significantly improves model alignment with the gold model.

TMU maintains classification accuracy after data removal.

Empirical results demonstrate superior performance over existing unlearning methods.

Abstract

Modern privacy regulations have spurred the evolution of machine unlearning, a technique that enables the removal of data from an already trained ML model without requiring retraining from scratch. Previous unlearning methods tend to induce the model to achieve lowest classification accuracy on the removal data. Nonetheless, the authentic objective of machine unlearning is to align the unlearned model with the gold model, i.e., achieving the same classification accuracy as the gold model. For this purpose, we present a Twin Machine Unlearning (TMU) approach, where a twin unlearning problem is defined corresponding to the original unlearning problem. As a results, the generalization-label predictor trained on the twin problem can be transferred to the original problem, facilitating aligned data removal. Comprehensive empirical experiments illustrate that our approach significantly enhances the alignment between the unlearned model and the gold model. Meanwhile, our method allows data removal without compromising the model accuracy.