Towards Dataset Copyright Evasion Attack against Personalized Text-to-Image Diffusion Models

TL;DR

This paper introduces CEAT2I, a novel attack method that effectively bypasses dataset ownership verification watermarks in personalized text-to-image diffusion models, highlighting vulnerabilities in current copyright protection techniques.

Contribution

The paper presents the first dataset copyright evasion attack (CEAT2I) targeting watermark-based ownership verification in T2I diffusion models, demonstrating its effectiveness against existing defenses.

Findings

CEAT2I successfully evades state-of-the-art DOV mechanisms.

Watermarks can be removed without degrading model performance.

Existing backdoor removal methods are less effective against CEAT2I.

Abstract

Text-to-image (T2I) diffusion models enable high-quality image generation conditioned on textual prompts. However, fine-tuning these pre-trained models for personalization raises concerns about unauthorized dataset usage. To address this issue, dataset ownership verification (DOV) has recently been proposed, which embeds watermarks into fine-tuning datasets via backdoor techniques. These watermarks remain dormant on benign samples but produce owner-specified outputs when triggered. Despite its promise, the robustness of DOV against copyright evasion attacks (CEA) remains unexplored. In this paper, we investigate how adversaries can circumvent these mechanisms, enabling models trained on watermarked datasets to bypass ownership verification. We begin by analyzing the limitations of potential attacks achieved by backdoor removal, including TPD and T2IShield. In practice, TPD suffers from inconsistent effectiveness due to randomness, while T2IShield fails when watermarks are embedded as local image patches. To this end, we introduce CEAT2I, the first CEA specifically targeting DOV in T2I diffusion models. CEAT2I consists of three stages: (1) motivated by the observation that T2I models converge faster on watermarked samples with respect to intermediate features rather than training loss, we reliably detect watermarked samples; (2) we iteratively ablate tokens from the prompts of detected samples and monitor feature shifts to identify trigger tokens; and (3) we apply a closed-form concept erasure method to remove the injected watermarks. Extensive experiments demonstrate that CEAT2I effectively evades state-of-the-art DOV mechanisms while preserving model performance. The code is available at https://github.com/csyufei/CEAT2I.