MarkPlugger: Generalizable Watermark Framework for Latent Diffusion Models without Retraining

TL;DR

MarkPlugger introduces a versatile, retraining-free watermarking framework for latent diffusion models that maintains image quality, is adaptable across different models, and resists various attacks, enhancing security in AI-generated content.

Contribution

The paper presents a novel plug-and-play watermarking method for LDMs that does not require retraining and is effective across multiple model variants and under attack.

Findings

Effective watermark embedding without modifying LDM components

High watermark recovery rate and preserved image quality

Robust against various attack methods

Abstract

Today, the family of latent diffusion models (LDMs) has gained prominence for its high quality outputs and scalability. This has also raised security concerns on social media, as malicious users can create and disseminate harmful content. Existing approaches typically involve training specific components or entire generative models to embed a watermark in generated images for traceability and responsibility. However, in the fast-evolving era of AI-generated content (AIGC), the rapid iteration and modification of LDMs makes retraining with watermark models costly. To address the problem, we propose MarkPlugger, a generalizable plug-and-play watermark framework without LDM retraining. In particular, to reduce the disturbance of the watermark on the semantics of the generated image, we try to identify a watermark representation that is approaching orthogonal to the semantic in latent space, and apply an additive fusion strategy for the watermark and the semantic. Without modifying any components of the LDMs, we embed diverse watermarks in latent space, adapting to the denoising process. Our experimental findings reveal that our method effectively harmonizes image quality and watermark recovery rate. We also have validated that our method is generalized to multiple official versions and modified variants of LDMs, even without retraining the watermark model. Furthermore, it performs robustly under various attacks of different intensities.