Memorized Images in Diffusion Models share a Subspace that can be Located and Deleted

TL;DR

This paper reveals that memorized images in diffusion models occupy a specific subspace and introduces a simple pruning method to remove this memorization, enhancing privacy and copyright protection without retraining.

Contribution

The study identifies a common subspace associated with memorized prompts and proposes a novel post-hoc pruning technique to mitigate memorization in diffusion models.

Findings

Memorized prompts activate a shared subspace in the model.

Pruning this subspace effectively reduces memorization.

Pruned models are more resistant to data extraction attacks.

Abstract

Large-scale text-to-image diffusion models excel in generating high-quality images from textual inputs, yet concerns arise as research indicates their tendency to memorize and replicate training data, raising We also addressed the issue of memorization in diffusion models, where models tend to replicate exact training samples raising copyright infringement and privacy issues. Efforts within the text-to-image community to address memorization explore causes such as data duplication, replicated captions, or trigger tokens, proposing per-prompt inference-time or training-time mitigation strategies. In this paper, we focus on the feed-forward layers and begin by contrasting neuron activations of a set of memorized and non-memorized prompts. Experiments reveal a surprising finding: many different sets of memorized prompts significantly activate a common subspace in the model, demonstrating, for the first time, that memorization in the diffusion models lies in a special subspace. Subsequently, we introduce a novel post-hoc method for editing pre-trained models, whereby memorization is mitigated through the straightforward pruning of weights in specialized subspaces, avoiding the need to disrupt the training or inference process as seen in prior research. Finally, we demonstrate the robustness of the pruned model against training data extraction attacks, thereby unveiling new avenues for a practical and one-for-all solution to memorization.