Secure and Efficient Watermarking for Latent Diffusion Models in Model Distribution Scenarios

TL;DR

This paper introduces a secure, efficient watermarking method for latent diffusion models that enhances copyright protection and robustness in model distribution scenarios, outperforming existing solutions against various attacks.

Contribution

A novel watermarking approach with a security mechanism that prevents leakage and escape, decoupled training for efficiency, and a verification strategy for robustness in model distribution.

Findings

Outperforms six baseline methods in effectiveness and robustness.

Resists ten types of image processing and adversarial attacks.

Enhances security during model distribution scenarios.

Abstract

Latent diffusion models have exhibited considerable potential in generative tasks. Watermarking is considered to be an alternative to safeguard the copyright of generative models and prevent their misuse. However, in the context of model distribution scenarios, the accessibility of models to large scale of model users brings new challenges to the security, efficiency and robustness of existing watermark solutions. To address these issues, we propose a secure and efficient watermarking solution. A new security mechanism is designed to prevent watermark leakage and watermark escape, which considers watermark randomness and watermark-model association as two constraints for mandatory watermark injection. To reduce the time cost of training the security module, watermark injection and the security mechanism are decoupled, ensuring that fine-tuning VAE only accomplishes the security mechanism without the burden of learning watermark patterns. A watermark distribution-based verification strategy is proposed to enhance the robustness against diverse attacks in the model distribution scenarios. Experimental results prove that our watermarking consistently outperforms existing six baselines on effectiveness and robustness against ten image processing attacks and adversarial attacks, while enhancing security in the distribution scenarios.