Learning to Unlearn for Robust Machine Unlearning

TL;DR

This paper introduces a novel meta-learning framework called Learning-to-Unlearn (LTU) that enhances machine unlearning by balancing data erasure and model performance through gradient harmonization.

Contribution

The paper proposes a new LTU framework with meta-optimization and gradient harmonization to improve unlearning efficiency and effectiveness, addressing key challenges in data rights and model reusability.

Findings

Improved unlearning efficiency and effectiveness.

Effective balance between forgetting specific data and retaining general knowledge.

Enhanced model performance after unlearning.

Abstract

Machine unlearning (MU) seeks to remove knowledge of specific data samples from trained models without the necessity for complete retraining, a task made challenging by the dual objectives of effective erasure of data and maintaining the overall performance of the model. Despite recent advances in this field, balancing between the dual objectives of unlearning remains challenging. From a fresh perspective of generalization, we introduce a novel Learning-to-Unlearn (LTU) framework, which adopts a meta-learning approach to optimize the unlearning process to improve forgetting and remembering in a unified manner. LTU includes a meta-optimization scheme that facilitates models to effectively preserve generalizable knowledge with only a small subset of the remaining set, while thoroughly forgetting the specific data samples. We also introduce a Gradient Harmonization strategy to align the optimization trajectories for remembering and forgetting via mitigating gradient conflicts, thus ensuring efficient and effective model updates. Our approach demonstrates improved efficiency and efficacy for MU, offering a promising solution to the challenges of data rights and model reusability.