Continuous Concepts Removal in Text-to-image Diffusion Models

TL;DR

This paper introduces CCRT, a novel method for continuous concept removal in text-to-image diffusion models that preserves image quality and text-image alignment during the removal process.

Contribution

The paper presents a new knowledge distillation approach with a genetic algorithm-based prompt generation to improve continuous concept removal in diffusion models.

Findings

Effective removal of targeted concepts demonstrated

Maintains high image quality and text-image alignment

Outperforms existing methods in experiments

Abstract

Text-to-image diffusion models have shown an impressive ability to generate high-quality images from input textual descriptions. However, concerns have been raised about the potential for these models to create content that infringes on copyrights or depicts disturbing subject matter. Removing specific concepts from these models is a promising potential solution to this problem. However, existing methods for concept removal do not work well in practical but challenging scenarios where concepts need to be continuously removed. Specifically, these methods lead to poor alignment between the text prompts and the generated image after the continuous removal process. To address this issue, we propose a novel approach called CCRT that includes a designed knowledge distillation paradigm. It constrains the text-image alignment behavior during the continuous concept removal process by using a set of text prompts generated through our genetic algorithm, which employs a designed fuzzing strategy. We conduct extensive experiments involving the removal of various concepts. The results evaluated through both algorithmic metrics and human studies demonstrate that our CCRT can effectively remove the targeted concepts in a continuous manner while maintaining the high generation quality (e.g., text-image alignment) of the model.