SHIFT: Stochastic Hidden-Trajectory Deflection for Removing Diffusion-based Watermark

TL;DR

SHIFT is a training-free attack that disrupts diffusion-based watermarks by deflecting the generative trajectory, achieving high success rates without degrading image quality.

Contribution

It introduces a novel stochastic resampling technique that effectively removes watermarks across diverse diffusion-based methods without prior knowledge.

Findings

Achieves 95-100% success rate in removing watermarks

Preserves semantic quality of images

Works across multiple watermarking paradigms

Abstract

Diffusion-based watermarking methods embed verifiable marks by manipulating the initial noise or the reverse diffusion trajectory. However, these methods share a critical assumption: verification can succeed only if the diffusion trajectory can be faithfully reconstructed. This reliance on trajectory recovery constitutes a fundamental and exploitable vulnerability. We propose $\underline{\mathbf{S}}$tochastic $\underline{\mathbf{Hi}}$dden-Trajectory De$\underline{\mathbf{f}}$lec$\underline{\mathbf{t}}$ion ($\mathbf{SHIFT}$), a training-free attack that exploits this common weakness across diverse watermarking paradigms. SHIFT leverages stochastic diffusion resampling to deflect the generative trajectory in latent space, making the reconstructed image statistically decoupled from the original watermark-embedded trajectory while preserving strong visual quality and semantic consistency. Extensive experiments on nine representative watermarking methods spanning noise-space, frequency-domain, and optimization-based paradigms show that SHIFT achieves 95%--100% attack success rates with nearly no loss in semantic quality, without requiring any watermark-specific knowledge or model retraining.