Fragile by Design: On the Limits of Adversarial Defenses in Personalized Generation

TL;DR

This paper critically examines the limitations of current adversarial defenses in personalized AI content generation, revealing their fragility and perceptibility issues, and introduces a framework to evaluate their robustness against realistic purification threats.

Contribution

The paper identifies key vulnerabilities in existing defenses like Anti-DreamBooth and proposes a systematic evaluation framework, AntiDB_Purify, to assess their robustness under realistic purification attacks.

Findings

Current defenses are easily detectable due to perceptible artifacts.

Perturbations are fragile and can be removed by simple filters.

Existing methods fail under realistic purification threats.

Abstract

Personalized AI applications such as DreamBooth enable the generation of customized content from user images, but also raise significant privacy concerns, particularly the risk of facial identity leakage. Recent defense mechanisms like Anti-DreamBooth attempt to mitigate this risk by injecting adversarial perturbations into user photos to prevent successful personalization. However, we identify two critical yet overlooked limitations of these methods. First, the adversarial examples often exhibit perceptible artifacts such as conspicuous patterns or stripes, making them easily detectable as manipulated content. Second, the perturbations are highly fragile, as even a simple, non-learned filter can effectively remove them, thereby restoring the model's ability to memorize and reproduce user identity. To investigate this vulnerability, we propose a novel evaluation framework, AntiDB_Purify, to systematically evaluate existing defenses under realistic purification threats, including both traditional image filters and adversarial purification. Results reveal that none of the current methods maintains their protective effectiveness under such threats. These findings highlight that current defenses offer a false sense of security and underscore the urgent need for more imperceptible and robust protections to safeguard user identity in personalized generation.