WMVLM: Evaluating Diffusion Model Image Watermarking via Vision-Language Models

TL;DR

This paper introduces WMVLM, a unified, interpretable framework for evaluating image watermarks in diffusion models using vision-language models, addressing previous limitations in metrics and security assessment.

Contribution

We propose WMVLM, the first comprehensive framework that evaluates both residual and semantic watermarks with interpretability and improved security metrics.

Findings

WMVLM outperforms existing VLMs in watermark evaluation tasks.

The framework generalizes well across datasets and diffusion models.

It provides interpretable assessments of watermark quality and security.

Abstract

Digital watermarking is essential for securing generated images from diffusion models. Accurate watermark evaluation is critical for algorithm development, yet existing methods have significant limitations: they lack a unified framework for both residual and semantic watermarks, provide results without interpretability, neglect comprehensive security considerations, and often use inappropriate metrics for semantic watermarks. To address these gaps, we propose WMVLM, the first unified and interpretable evaluation framework for diffusion model image watermarking via vision-language models (VLMs). We redefine quality and security metrics for each watermark type: residual watermarks are evaluated by artifact strength and erasure resistance, while semantic watermarks are assessed through latent distribution shifts. Moreover, we introduce a three-stage training strategy to progressively enable the model to achieve classification, scoring, and interpretable text generation. Experiments show WMVLM outperforms state-of-the-art VLMs with strong generalization across datasets, diffusion models, and watermarking methods.