SUGAR: A Sweeter Spot for Generative Unlearning of Many Identities

TL;DR

SUGAR is a scalable framework that unlearns specific identities from 3D-aware generative models efficiently, preserving model quality and utility while removing up to 200 identities without retraining.

Contribution

It introduces a novel personalized surrogate latent approach and a continual utility preservation objective for effective generative unlearning of multiple identities.

Findings

Removes up to 200 identities effectively.

Achieves 700% utility retention improvement.

Operates without retraining the entire model.

Abstract

Recent advances in 3D-aware generative models have enabled high-fidelity image synthesis of human identities. However, this progress raises urgent questions around user consent and the ability to remove specific individuals from a model's output space. We address this by introducing SUGAR, a framework for scalable generative unlearning that enables the removal of many identities (simultaneously or sequentially) without retraining the entire model. Rather than projecting unwanted identities to unrealistic outputs or relying on static template faces, SUGAR learns a personalized surrogate latent for each identity, diverting reconstructions to visually coherent alternatives while preserving the model's quality and diversity. We further introduce a continual utility preservation objective that guards against degradation as more identities are forgotten. SUGAR achieves state-of-the-art performance in removing up to 200 identities, while delivering up to a 700% improvement in retention utility compared to existing baselines. Our code is publicly available at https://github.com/judydnguyen/SUGAR-Generative-Unlearn.