One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework

TL;DR

This paper introduces Co-Erasing, a novel framework that enhances concept erasure in text-to-image models by integrating visual supervision and text-guided refinement, significantly improving efficacy while preserving usability.

Contribution

The paper proposes the first text-image collaborative erasing framework that directly incorporates visual supervision to overcome modality gaps, improving concept removal effectiveness.

Findings

Outperforms state-of-the-art erasure methods in efficacy.

Achieves a better balance between erasure effectiveness and usability.

Effectively reduces undesirable concept generation in diffusion models.

Abstract

Concept erasing has recently emerged as an effective paradigm to prevent text-to-image diffusion models from generating visually undesirable or even harmful content. However, current removal methods heavily rely on manually crafted text prompts, making it challenging to achieve a high erasure (efficacy) while minimizing the impact on other benign concepts (usability). In this paper, we attribute the limitations to the inherent gap between the text and image modalities, which makes it hard to transfer the intricately entangled concept knowledge from text prompts to the image generation process. To address this, we propose a novel solution by directly integrating visual supervision into the erasure process, introducing the first text-image Collaborative Concept Erasing (Co-Erasing) framework. Specifically, Co-Erasing describes the concept jointly by text prompts and the corresponding undesirable images induced by the prompts, and then reduces the generating probability of the target concept through negative guidance. This approach effectively bypasses the knowledge gap between text and image, significantly enhancing erasure efficacy. Additionally, we design a text-guided image concept refinement strategy that directs the model to focus on visual features most relevant to the specified text concept, minimizing disruption to other benign concepts. Finally, comprehensive experiments suggest that Co-Erasing outperforms state-of-the-art erasure approaches significantly with a better trade-off between efficacy and usability. Codes are available at https://github.com/Ferry-Li/Co-Erasing.