Exploiting Watermark-Based Defense Mechanisms in Text-to-Image Diffusion Models for Unauthorized Data Usage

TL;DR

This paper introduces RATTAN, a novel method leveraging the diffusion process to enhance watermark robustness in text-to-image models, effectively protecting against unauthorized data usage and surpassing existing defenses.

Contribution

The paper proposes RATTAN, a diffusion-based approach that improves watermark robustness in text-to-image models, addressing limitations of current protection methods.

Findings

Existing protections are vulnerable to RATTAN.

RATTAN effectively preserves high-level features during image generation.

Experiments show RATTAN outperforms state-of-the-art defenses.

Abstract

Text-to-image diffusion models, such as Stable Diffusion, have shown exceptional potential in generating high-quality images. However, recent studies highlight concerns over the use of unauthorized data in training these models, which may lead to intellectual property infringement or privacy violations. A promising approach to mitigate these issues is to apply a watermark to images and subsequently check if generative models reproduce similar watermark features. In this paper, we examine the robustness of various watermark-based protection methods applied to text-to-image models. We observe that common image transformations are ineffective at removing the watermark effect. Therefore, we propose RATTAN, that leverages the diffusion process to conduct controlled image generation on the protected input, preserving the high-level features of the input while ignoring the low-level details utilized by watermarks. A small number of generated images are then used to fine-tune protected models. Our experiments on three datasets and 140 text-to-image diffusion models reveal that existing state-of-the-art protections are not robust against RATTAN.