Zero-Shot Machine Unlearning with Proxy Adversarial Data Generation

TL;DR

This paper introduces ZS-PAG, a zero-shot machine unlearning framework that generates proxy adversarial data to effectively remove specific samples' influence without access to remaining data, ensuring model performance.

Contribution

The paper proposes a novel zero-shot unlearning method using adversarial data generation and influence-based pseudo-labeling, addressing practical scenarios with limited data access.

Findings

Outperforms existing unlearning methods on multiple benchmarks.

Effectively prevents over-unlearning in zero-shot scenarios.

Provides theoretical guarantees for the unlearning process.

Abstract

Machine unlearning aims to remove the influence of specific samples from a trained model. A key challenge in this process is over-unlearning, where the model's performance on the remaining data significantly drops due to the change in the model's parameters. Existing unlearning algorithms depend on the remaining data to prevent this issue. As such, these methods are inapplicable in a more practical scenario, where only the unlearning samples are available (i.e., zero-shot unlearning). This paper presents a novel framework, ZS-PAG, to fill this gap. Our approach offers three key innovations: (1) we approximate the inaccessible remaining data by generating adversarial samples; (2) leveraging the generated samples, we pinpoint a specific subspace to perform the unlearning process, therefore preventing over-unlearning in the challenging zero-shot scenario; and (3) we consider the influence of the unlearning process on the remaining samples and design an influence-based pseudo-labeling strategy. As a result, our method further improves the model's performance after unlearning. The proposed method holds a theoretical guarantee, and experiments on various benchmarks validate the effectiveness and superiority of our proposed method over several baselines.