HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Models

TL;DR

HMARK introduces a multi-bit semantic-latent watermarking scheme for diffusion models that ensures high detection accuracy, robustness, and minimal perceptual impact, aiding in ownership verification of generated images.

Contribution

It proposes a novel multi-bit watermarking method in semantic-latent space, enhancing robustness and interpretability for diffusion model outputs.

Findings

Achieves 98.57% watermark detection accuracy.

Attains 95.07% bit-level recovery accuracy.

Maintains 100% recall rate and 1.0 AUC across distortions.

Abstract

Modern generative diffusion models rely on vast training datasets, often including images with uncertain ownership or usage rights. Radioactive watermarks -- marks that transfer to a model's outputs -- can help detect when such unauthorized data has been used for training. Moreover, aside from being radioactive, an effective watermark for protecting images from unauthorized training also needs to meet other existing requirements, such as imperceptibility, robustness, and multi-bit capacity. To overcome these challenges, we propose HMARK, a novel multi-bit watermarking scheme, which encodes ownership information as secret bits in the semantic-latent space (h-space) for image diffusion models. By leveraging the interpretability and semantic significance of h-space, ensuring that watermark signals correspond to meaningful semantic attributes, the watermarks embedded by HMARK exhibit radioactivity, robustness to distortions, and minimal impact on perceptual quality. Experimental results demonstrate that HMARK achieves 98.57% watermark detection accuracy, 95.07% bit-level recovery accuracy, 100% recall rate, and 1.0 AUC on images produced by the downstream adversarial model finetuned with LoRA on watermarked data across various types of distortions.