Watermarking Visual Concepts for Diffusion Models

TL;DR

This paper introduces ConceptWM, a lightweight and robust watermarking framework for diffusion models that enables simultaneous concept identification and model tracing, enhancing copyright protection and resisting adversarial attacks.

Contribution

It proposes a novel single-stage concept watermarking method with adversarial perturbation injection, outperforming existing techniques in accuracy and robustness against model fine-tuning.

Findings

Improves concept detection accuracy by 6.3%-19.3%.

Maintains 21.7% FID/CLIP degradation under adversarial fine-tuning.

Outperforms state-of-the-art watermarking methods on multiple datasets.

Abstract

The personalization techniques of diffusion models succeed in generating images with specific concepts. This ability also poses great threats to copyright protection and network security since malicious users can generate unauthorized content and disinformation relevant to a target concept. Model watermarking is an effective solution to trace the malicious generated images and safeguard their copyright. However, existing model watermarking techniques merely achieve image-level tracing without concept traceability. When tracing infringing or harmful concepts, current approaches execute image concept detection and model tracing sequentially, where performance is critically constrained by concept detection accuracy. In this paper, we propose a lightweight concept watermarking framework that efficiently binds target concepts to model watermarks, supporting simultaneous concept identification and model tracing via single-stage watermark verification. To further enhance the robustness of concept watermarking, we propose an adversarial perturbation injection method collaboratively embedded with watermarks during image generation, avoiding watermark removal by model purification attacks. Experimental results demonstrate that ConceptWM significantly outperforms state-of-the-art watermarking methods, improving detection accuracy by 6.3%-19.3% across diverse datasets including COCO and StableDiffusionDB. Additionally, ConceptWM possesses a critical capability absent in other watermarking methods: it sustains a 21.7% FID/CLIP degradation under adversarial fine-tuning of Stable Diffusion models on WikiArt and CelebA-HQ, demonstrating its capability to mitigate model misuse.