Efficient Unlearning through Maximizing Relearning Convergence Delay

TL;DR

This paper introduces a new metric called relearning convergence delay for evaluating machine unlearning, and proposes an influence eliminating framework that improves unlearning effectiveness while maintaining model accuracy.

Contribution

The paper presents a novel metric for comprehensive unlearning assessment and a framework that effectively removes influence of forgotten data with theoretical guarantees.

Findings

The new metric outperforms existing evaluation metrics.

The proposed framework achieves strong retention and resistance to relearning.

Theoretical guarantees include exponential convergence and upper bounds.

Abstract

Machine unlearning poses challenges in removing mislabeled, contaminated, or problematic data from a pretrained model. Current unlearning approaches and evaluation metrics are solely focused on model predictions, which limits insight into the model's true underlying data characteristics. To address this issue, we introduce a new metric called relearning convergence delay, which captures both changes in weight space and prediction space, providing a more comprehensive assessment of the model's understanding of the forgotten dataset. This metric can be used to assess the risk of forgotten data being recovered from the unlearned model. Based on this, we propose the Influence Eliminating Unlearning framework, which removes the influence of the forgetting set by degrading its performance and incorporates weight decay and injecting noise into the model's weights, while maintaining accuracy on the retaining set. Extensive experiments show that our method outperforms existing metrics and our proposed relearning convergence delay metric, approaching ideal unlearning performance. We provide theoretical guarantees, including exponential convergence and upper bounds, as well as empirical evidence of strong retention and resistance to relearning in both classification and generative unlearning tasks.