Erasing More Than Intended? How Concept Erasure Degrades the Generation of Non-Target Concepts

TL;DR

This paper investigates how concept erasure in text-to-image models can unintentionally harm non-target concepts, revealing the phenomenon of concept entanglement and introducing EraseBench for comprehensive evaluation.

Contribution

It identifies evaluation gaps in concept erasure methods, analyzes failure modes, and introduces EraseBench, a benchmark for assessing post-erasure effects on diverse concepts.

Findings

Concept erasure causes unintended suppression of non-target concepts.

EraseBench effectively evaluates erasure effectiveness and side effects.

Concept entanglement leads to spillover degradation in generated images.

Abstract

Concept erasure techniques have recently gained significant attention for their potential to remove unwanted concepts from text-to-image models. While these methods often demonstrate promising results in controlled settings, their robustness in real-world applications and suitability for deployment remain uncertain. In this work, we (1) identify a critical gap in evaluating sanitized models, particularly in assessing their performance across diverse concept dimensions, and (2) systematically analyze the failure modes of text-to-image models post-erasure. We focus on the unintended consequences of concept removal on non-target concepts across different levels of interconnected relationships including visually similar, binomial, and semantically related concepts. To address this, we introduce EraseBench, a comprehensive benchmark for evaluating post-erasure performance. EraseBench includes over 100 curated concepts, targeted evaluation prompts, and a robust set of metrics to assess both effectiveness and side effects of erasure. Our findings reveal a phenomenon of concept entanglement, where erasure leads to unintended suppression of non-target concepts, causing spillover degradation that manifests as distortions and a decline in generation quality.