VideoMarkBench: Benchmarking Robustness of Video Watermarking

TL;DR

VideoMarkBench provides a comprehensive benchmark for assessing the robustness of video watermarking methods against various attacks, revealing significant vulnerabilities and emphasizing the need for more resilient solutions in the context of synthetic videos.

Contribution

This work introduces the first systematic benchmark for evaluating video watermarking robustness across multiple models, styles, and attack types, filling a critical gap in the field.

Findings

Current watermarking methods are vulnerable to various perturbations.

Robustness varies significantly across different watermarking techniques.

The benchmark highlights the urgent need for developing more resilient watermarking solutions.

Abstract

The rapid development of video generative models has led to a surge in highly realistic synthetic videos, raising ethical concerns related to disinformation and copyright infringement. Recently, video watermarking has been proposed as a mitigation strategy by embedding invisible marks into AI-generated videos to enable subsequent detection. However, the robustness of existing video watermarking methods against both common and adversarial perturbations remains underexplored. In this work, we introduce VideoMarkBench, the first systematic benchmark designed to evaluate the robustness of video watermarks under watermark removal and watermark forgery attacks. Our study encompasses a unified dataset generated by three state-of-the-art video generative models, across three video styles, incorporating four watermarking methods and seven aggregation strategies used during detection. We comprehensively evaluate 12 types of perturbations under white-box, black-box, and no-box threat models. Our findings reveal significant vulnerabilities in current watermarking approaches and highlight the urgent need for more robust solutions. Our code is available at https://github.com/zhengyuan-jiang/VideoMarkBench.