SafeEditor: Unified MLLM for Efficient Post-hoc T2I Safety Editing

TL;DR

SafeEditor introduces a unified multi-round safety editing framework using a multi-modal large language model, significantly improving safety alignment in text-to-image models while maintaining utility and reducing over-refusal.

Contribution

The paper presents a novel post-hoc safety editing paradigm and SafeEditor, a multi-modal LLM that enables efficient, multi-round safety editing for any T2I model, advancing safety without sacrificing utility.

Findings

SafeEditor reduces over-refusal compared to prior methods.

It achieves a better safety-utility balance in T2I models.

Experimental results demonstrate superior safety editing performance.

Abstract

With the rapid advancement of text-to-image (T2I) models, ensuring their safety has become increasingly critical. Existing safety approaches can be categorized into training-time and inference-time methods. While inference-time methods are widely adopted due to their cost-effectiveness, they often suffer from limitations such as over-refusal and imbalance between safety and utility. To address these challenges, we propose a multi-round safety editing framework that functions as a model-agnostic, plug-and-play module, enabling efficient safety alignment for any text-to-image model. Central to this framework is MR-SafeEdit, a multi-round image-text interleaved dataset specifically constructed for safety editing in text-to-image generation. We introduce a post-hoc safety editing paradigm that mirrors the human cognitive process of identifying and refining unsafe content. To instantiate this paradigm, we develop SafeEditor, a unified MLLM capable of multi-round safety editing on generated images. Experimental results show that SafeEditor surpasses prior safety approaches by reducing over-refusal while achieving a more favorable safety-utility balance.