IP-Bench: Benchmark for Image Protection Methods in Image-to-Video Generation Scenarios

TL;DR

IP-Bench is a comprehensive benchmark designed to evaluate image protection methods specifically in image-to-video generation scenarios, addressing the lack of unified evaluation frameworks and assessing robustness against attacks.

Contribution

It introduces the first systematic benchmark for evaluating protection methods in I2V scenarios, including robustness and transferability assessments across models and modalities.

Findings

Evaluates 6 protection methods against 5 I2V models.

Assesses robustness with two attack strategies in practical scenarios.

Analyzes cross-model and cross-modality transferability of protection methods.

Abstract

With the rapid advancement of image-to-video (I2V) generation models, their potential for misuse in creating malicious content has become a significant concern. For instance, a single image can be exploited to generate a fake video, which can be used to attract attention and gain benefits. This phenomenon is referred to as an I2V generation misuse. Existing image protection methods suffer from the absence of a unified benchmark, leading to an incomplete evaluation framework. Furthermore, these methods have not been systematically assessed in I2V generation scenarios and against preprocessing attacks, which complicates the evaluation of their effectiveness in real-world deployment scenarios.To address this challenge, we propose IP-Bench (Image Protection Bench), the first systematic benchmark designed to evaluate protection methods in I2V generation scenarios. This benchmark examines 6 representative protection methods and 5 state-of-the-art I2V models. Furthermore, our work systematically evaluates protection methods' robustness with two robustness attack strategies under practical scenarios and analyzes their cross-model & cross-modality transferability. Overall, IP-Bench establishes a systematic, reproducible, and extensible evaluation framework for image protection methods in I2V generation scenarios.