AEGIS: Adversarial Target-Guided Retention-Data-Free Robust Concept Erasure from Diffusion Models

TL;DR

AEGIS is a novel framework for concept erasure in diffusion models that enhances robustness against reactivation and preserves unrelated concepts without requiring retention data, addressing key challenges in safe model fine-tuning.

Contribution

The paper introduces AEGIS, a retention-data-free adversarial framework that simultaneously improves robustness and retention in concept erasure for diffusion models.

Findings

AEGIS outperforms existing methods in robustness against reactivation.

AEGIS maintains better retention of unrelated concepts.

Experimental results demonstrate superior safety and utility balance.

Abstract

Concept erasure helps stop diffusion models (DMs) from generating harmful content; but current methods face robustness retention trade off. Robustness means the model fine-tuned by concept erasure methods resists reactivation of erased concepts, even under semantically related prompts. Retention means unrelated concepts are preserved so the model's overall utility stays intact. Both are critical for concept erasure in practice, yet addressing them simultaneously is challenging, as existing works typically improve one factor while sacrificing the other. Prior work typically strengthens one while degrading the other, e.g., mapping a single erased prompt to a fixed safe target leaves class level remnants exploitable by prompt attacks, whereas retention-oriented schemes underperform against adaptive adversaries. This paper introduces Adversarial Erasure with Gradient Informed Synergy (AEGIS), a retention-data-free framework that advances both robustness and retention.