AlignGuard: Scalable Safety Alignment for Text-to-Image Generation

TL;DR

AlignGuard introduces a scalable safety alignment method for text-to-image models using synthetic datasets and expert merging, significantly reducing harmful content generation and outperforming existing safety measures.

Contribution

The paper presents a novel expert-based safety alignment approach for T2I models using DPO and a new merging strategy, enabling removal of more harmful concepts than previous methods.

Findings

Removes 7x more harmful concepts than baselines

Outperforms state-of-the-art safety benchmarks

Scalable safety alignment via expert merging

Abstract

Text-to-image (T2I) models are widespread, but their limited safety guardrails expose end users to harmful content and potentially allow for model misuse. Current safety measures are typically limited to text-based filtering or concept removal strategies, able to remove just a few concepts from the model's generative capabilities. In this work, we introduce AlignGuard, a method for safety alignment of T2I models. We enable the application of Direct Preference Optimization (DPO) for safety purposes in T2I models by synthetically generating a dataset of harmful and safe image-text pairs, which we call CoProV2. Using a custom DPO strategy and this dataset, we train safety experts, in the form of low-rank adaptation (LoRA) matrices, able to guide the generation process away from specific safety-related concepts. Then, we merge the experts into a single LoRA using a novel merging strategy for optimal scaling performance. This expert-based approach enables scalability, allowing us to remove 7x more harmful concepts from T2I models compared to baselines. AlignGuard consistently outperforms the state-of-the-art on many benchmarks and establishes new practices for safety alignment in T2I networks. Code and data will be shared at https://safetydpo.github.io/.