Machine Unlearning via Null Space Calibration

TL;DR

This paper introduces UNSC, a novel machine unlearning method that precisely forgets specific data without degrading overall model performance, by calibrating the decision space within a null space framework.

Contribution

The paper proposes a null space calibration approach for machine unlearning that prevents over-unlearning and maintains model accuracy on remaining data.

Findings

UNSC effectively unlearns target samples without over-unlearning.

The approach improves model performance on remaining data after unlearning.

Comparative results show UNSC outperforms existing baselines.

Abstract

Machine unlearning aims to enable models to forget specific data instances when receiving deletion requests. Current research centres on efficient unlearning to erase the influence of data from the model and neglects the subsequent impacts on the remaining data. Consequently, existing unlearning algorithms degrade the model's performance after unlearning, known as \textit{over-unlearning}. This paper addresses this critical yet under-explored issue by introducing machine \underline{U}nlearning via \underline{N}ull \underline{S}pace \underline{C}alibration (UNSC), which can accurately unlearn target samples without over-unlearning. On the contrary, by calibrating the decision space during unlearning, UNSC can significantly improve the model's performance on the remaining samples. In particular, our approach hinges on confining the unlearning process to a specified null space tailored to the remaining samples, which is augmented by strategically pseudo-labeling the unlearning samples. Comparative analyses against several established baselines affirm the superiority of our approach. Code is released at this \href{https://github.com/HQC-ML/Machine-Unlearning-via-Null-Space-Calibration}{URL}.