JIGMARK: A Black-Box Approach for Enhancing Image Watermarks against Diffusion Model Edits

TL;DR

JIGMARK is a novel black-box watermarking method that significantly improves robustness against diffusion model edits and traditional perturbations without requiring gradient access, using contrastive learning.

Contribution

It introduces JIGMARK, a contrastive learning-based watermarking technique that enhances robustness against diffusion model edits without backpropagation, and proposes the HAV score for better robustness measurement.

Findings

JIGMARK achieves over three times higher true positive rate than baselines at 1% false positive rate.

It outperforms existing watermarking methods in resisting diffusion model edits and conventional perturbations.

The HAV score provides a more effective metric for quantifying image derivatives after editing.

Abstract

In this study, we investigate the vulnerability of image watermarks to diffusion-model-based image editing, a challenge exacerbated by the computational cost of accessing gradient information and the closed-source nature of many diffusion models. To address this issue, we introduce JIGMARK. This first-of-its-kind watermarking technique enhances robustness through contrastive learning with pairs of images, processed and unprocessed by diffusion models, without needing a direct backpropagation of the diffusion process. Our evaluation reveals that JIGMARK significantly surpasses existing watermarking solutions in resilience to diffusion-model edits, demonstrating a True Positive Rate more than triple that of leading baselines at a 1% False Positive Rate while preserving image quality. At the same time, it consistently improves the robustness against other conventional perturbations (like JPEG, blurring, etc.) and malicious watermark attacks over the state-of-the-art, often by a large margin. Furthermore, we propose the Human Aligned Variation (HAV) score, a new metric that surpasses traditional similarity measures in quantifying the number of image derivatives from image editing.