Cross-Modal Transferable Image-to-Video Attack on Video Quality Metrics

TL;DR

This paper introduces IC2VQA, a cross-modal attack method that exploits similarities in low-level features between images and videos to effectively generate adversarial examples that fool black-box VQA models, revealing vulnerabilities in current metrics.

Contribution

The paper proposes a novel cross-modal attack approach, IC2VQA, which leverages a CLIP module to improve transferability of adversarial perturbations across different VQA models.

Findings

IC2VQA achieves high success rates in attacking black-box VQA models.

The method outperforms existing black-box attack strategies in efficiency and effectiveness.

Adding a CLIP module enhances transferability of adversarial examples.

Abstract

Recent studies have revealed that modern image and video quality assessment (IQA/VQA) metrics are vulnerable to adversarial attacks. An attacker can manipulate a video through preprocessing to artificially increase its quality score according to a certain metric, despite no actual improvement in visual quality. Most of the attacks studied in the literature are white-box attacks, while black-box attacks in the context of VQA have received less attention. Moreover, some research indicates a lack of transferability of adversarial examples generated for one model to another when applied to VQA. In this paper, we propose a cross-modal attack method, IC2VQA, aimed at exploring the vulnerabilities of modern VQA models. This approach is motivated by the observation that the low-level feature spaces of images and videos are similar. We investigate the transferability of adversarial perturbations across different modalities; specifically, we analyze how adversarial perturbations generated on a white-box IQA model with an additional CLIP module can effectively target a VQA model. The addition of the CLIP module serves as a valuable aid in increasing transferability, as the CLIP model is known for its effective capture of low-level semantics. Extensive experiments demonstrate that IC2VQA achieves a high success rate in attacking three black-box VQA models. We compare our method with existing black-box attack strategies, highlighting its superiority in terms of attack success within the same number of iterations and levels of attack strength. We believe that the proposed method will contribute to the deeper analysis of robust VQA metrics.