Synthetic Forgetting without Access: A Few-shot Zero-glance Framework for Machine Unlearning

TL;DR

This paper presents GFOES, a framework that enables effective machine unlearning in data-constrained scenarios by synthesizing erasure samples without access to original forget data, ensuring privacy and model utility.

Contribution

Introducing GFOES, a novel few-shot zero-glance unlearning framework that synthesizes erasure samples for effective forgetting without access to the forget set.

Findings

Achieves effective forgetting on image classification datasets.

Maintains strong model performance with only 5% of original data.

Operates effectively at both logit and representation levels.

Abstract

Machine unlearning aims to eliminate the influence of specific data from trained models to ensure privacy compliance. However, most existing methods assume full access to the original training dataset, which is often impractical. We address a more realistic yet challenging setting: few-shot zero-glance, where only a small subset of the retained data is available and the forget set is entirely inaccessible. We introduce GFOES, a novel framework comprising a Generative Feedback Network (GFN) and a two-phase fine-tuning procedure. GFN synthesises Optimal Erasure Samples (OES), which induce high loss on target classes, enabling the model to forget class-specific knowledge without access to the original forget data, while preserving performance on retained classes. The two-phase fine-tuning procedure enables aggressive forgetting in the first phase, followed by utility restoration in the second. Experiments on three image classification datasets demonstrate that GFOES achieves effective forgetting at both logit and representation levels, while maintaining strong performance using only 5% of the original data. Our framework offers a practical and scalable solution for privacy-preserving machine learning under data-constrained conditions.