Robust Continual Unlearning against Knowledge Erosion and Forgetting Reversal

TL;DR

This paper introduces SAFER, a continual unlearning framework that effectively mitigates knowledge erosion and forgetting reversal, ensuring stable AI model performance over multiple unlearning phases.

Contribution

The paper proposes SAFER, a novel continual unlearning method that maintains stability and prevents forgetting reversal in iterative unlearning scenarios.

Findings

SAFER reduces accuracy degradation on retain data over multiple unlearning phases.

SAFER prevents previously forgotten data from being recognized again in later phases.

Experiments demonstrate SAFER's superior stability compared to existing methods.

Abstract

As a means to balance the growth of the AI industry with the need for privacy protection, machine unlearning plays a crucial role in realizing the ``right to be forgotten'' in artificial intelligence. This technique enables AI systems to remove the influence of specific data while preserving the rest of the learned knowledge. Although it has been actively studied, most existing unlearning methods assume that unlearning is performed only once. In this work, we evaluate existing unlearning algorithms in a more realistic scenario where unlearning is conducted repeatedly, and in this setting, we identify two critical phenomena: (1) Knowledge Erosion, where the accuracy on retain data progressively degrades over unlearning phases, and (2) Forgetting Reversal, where previously forgotten samples become recognizable again in later phases. To address these challenges, we propose SAFER (StAbility-preserving Forgetting with Effective Regularization), a continual unlearning framework that maintains representation stability for retain data while enforcing negative logit margins for forget data. Extensive experiments show that SAFER mitigates not only knowledge erosion but also forgetting reversal, achieving stable performance across multiple unlearning phases.