Towards Source-Free Machine Unlearning

TL;DR

This paper introduces a novel source-free machine unlearning method that removes specific data from trained models without access to original training data, using Hessian estimation for efficient and theoretically guaranteed unlearning.

Contribution

We propose a new source-free unlearning approach that estimates the Hessian of unknown data, enabling efficient zero-shot data removal with theoretical guarantees.

Findings

Effective unlearning demonstrated across multiple datasets.

Maintains model performance on remaining data.

Provides theoretical guarantees for unlearning quality.

Abstract

As machine learning becomes more pervasive and data privacy regulations evolve, the ability to remove private or copyrighted information from trained models is becoming an increasingly critical requirement. Existing unlearning methods often rely on the assumption of having access to the entire training dataset during the forgetting process. However, this assumption may not hold true in practical scenarios where the original training data may not be accessible, i.e., the source-free setting. To address this challenge, we focus on the source-free unlearning scenario, where an unlearning algorithm must be capable of removing specific data from a trained model without requiring access to the original training dataset. Building on recent work, we present a method that can estimate the Hessian of the unknown remaining training data, a crucial component required for efficient unlearning. Leveraging this estimation technique, our method enables efficient zero-shot unlearning while providing robust theoretical guarantees on the unlearning performance, while maintaining performance on the remaining data. Extensive experiments over a wide range of datasets verify the efficacy of our method.