Safety Without Semantic Disruptions: Editing-free Safe Image Generation via Context-preserving Dual Latent Reconstruction

TL;DR

This paper introduces a novel diffusion-based method that enhances safety in image generation by preserving semantic integrity and offering controllable safety levels, addressing limitations of existing model editing techniques.

Contribution

The authors propose a safe embedding and modified diffusion process that maintains semantic structure while improving safety, outperforming prior methods on safety benchmarks.

Findings

State-of-the-art safety in image generation benchmarks

Preserves semantic structure during safety modifications

Offers intuitive control over safety levels

Abstract

Training multimodal generative models on large, uncurated datasets can result in users being exposed to harmful, unsafe and controversial or culturally-inappropriate outputs. While model editing has been proposed to remove or filter undesirable concepts in embedding and latent spaces, it can inadvertently damage learned manifolds, distorting concepts in close semantic proximity. We identify limitations in current model editing techniques, showing that even benign, proximal concepts may become misaligned. To address the need for safe content generation, we leverage safe embeddings and a modified diffusion process with tunable weighted summation in the latent space to generate safer images. Our method preserves global context without compromising the structural integrity of the learned manifolds. We achieve state-of-the-art results on safe image generation benchmarks and offer intuitive control over the level of model safety. We identify trade-offs between safety and censorship, which presents a necessary perspective in the development of ethical AI models. We will release our code. Keywords: Text-to-Image Models, Generative AI, Safety, Reliability, Model Editing