LURE: Latent Space Unblocking for Multi-Concept Reawakening in Diffusion Models

TL;DR

This paper introduces LURE, a novel method for reawakening erased concepts in diffusion models by reconstructing latent space and guiding sampling, addressing limitations of prompt-based approaches.

Contribution

LURE offers a comprehensive theoretical framework and a practical technique for reawakening multiple erased concepts in diffusion models, improving over existing prompt-based methods.

Findings

LURE achieves high-fidelity reawakening of multiple concepts.

It outperforms existing methods across diverse erasure tasks.

Theoretical analysis confirms perturbation of various factors can reawaken concepts.

Abstract

Concept erasure aims to suppress sensitive content in diffusion models, but recent studies show that erased concepts can still be reawakened, revealing vulnerabilities in erasure methods. Existing reawakening methods mainly rely on prompt-level optimization to manipulate sampling trajectories, neglecting other generative factors, which limits a comprehensive understanding of the underlying dynamics. In this paper, we model the generation process as an implicit function to enable a comprehensive theoretical analysis of multiple factors, including text conditions, model parameters, and latent states. We theoretically show that perturbing each factor can reawaken erased concepts. Building on this insight, we propose a novel concept reawakening method: Latent space Unblocking for concept REawakening (LURE), which reawakens erased concepts by reconstructing the latent space and guiding the sampling trajectory. Specifically, our semantic re-binding mechanism reconstructs the latent space by aligning denoising predictions with target distributions to reestablish severed text-visual associations. However, in multi-concept scenarios, naive reconstruction can cause gradient conflicts and feature entanglement. To address this, we introduce Gradient Field Orthogonalization, which enforces feature orthogonality to prevent mutual interference. Additionally, our Latent Semantic Identification-Guided Sampling (LSIS) ensures stability of the reawakening process via posterior density verification. Extensive experiments demonstrate that LURE enables simultaneous, high-fidelity reawakening of multiple erased concepts across diverse erasure tasks and methods.