Trigger-Based Fragile Model Watermarking for Image Transformation Networks

TL;DR

This paper presents a novel trigger-based fragile watermarking method for image transformation and generation networks, enabling source verification and integrity checking by embedding visual patterns that break upon tampering.

Contribution

It introduces the first fragile watermarking approach tailored for image transformation/generation models using trigger-based visual patterns.

Findings

Outperforms baseline methods by 94% in verification accuracy.

Effective across various datasets and attack scenarios.

Applicable to multiple image transformation tasks.

Abstract

In fragile watermarking, a sensitive watermark is embedded in an object in a manner such that the watermark breaks upon tampering. This fragile process can be used to ensure the integrity and source of watermarked objects. While fragile watermarking for model integrity has been studied in classification models, image transformation/generation models have yet to be explored. We introduce a novel, trigger-based fragile model watermarking system for image transformation/generation networks that takes advantage of properties inherent to image outputs. For example, manifesting watermarks as specific visual patterns, styles, or anomalies in the generated content when particular trigger inputs are used. Our approach, distinct from robust watermarking, effectively verifies the model's source and integrity across various datasets and attacks, outperforming baselines by 94%. We conduct additional experiments to analyze the security of this approach, the flexibility of the trigger and resulting watermark, and the sensitivity of the watermarking loss on performance. We also demonstrate the applicability of this approach on two different tasks (1 immediate task and 1 downstream task). This is the first work to consider fragile model watermarking for image transformation/generation networks.