ReToMe-VA: Recursive Token Merging for Video Diffusion-based Unrestricted Adversarial Attack

TL;DR

ReToMe-VA is a novel framework for generating imperceptible, highly transferable adversarial video clips using recursive token merging and latent space optimization within diffusion models, advancing video attack methods.

Contribution

It introduces the first recursive token merging mechanism for temporally consistent adversarial videos and a Timestep-wise Adversarial Latent Optimization strategy for improved imperceptibility and transferability.

Findings

Outperforms state-of-the-art attacks in transferability by over 14%.

Achieves high imperceptibility in both spatial and temporal domains.

Demonstrates effectiveness across various video datasets.

Abstract

Recent diffusion-based unrestricted attacks generate imperceptible adversarial examples with high transferability compared to previous unrestricted attacks and restricted attacks. However, existing works on diffusion-based unrestricted attacks are mostly focused on images yet are seldom explored in videos. In this paper, we propose the Recursive Token Merging for Video Diffusion-based Unrestricted Adversarial Attack (ReToMe-VA), which is the first framework to generate imperceptible adversarial video clips with higher transferability. Specifically, to achieve spatial imperceptibility, ReToMe-VA adopts a Timestep-wise Adversarial Latent Optimization (TALO) strategy that optimizes perturbations in diffusion models' latent space at each denoising step. TALO offers iterative and accurate updates to generate more powerful adversarial frames. TALO can further reduce memory consumption in gradient computation. Moreover, to achieve temporal imperceptibility, ReToMe-VA introduces a Recursive Token Merging (ReToMe) mechanism by matching and merging tokens across video frames in the self-attention module, resulting in temporally consistent adversarial videos. ReToMe concurrently facilitates inter-frame interactions into the attack process, inducing more diverse and robust gradients, thus leading to better adversarial transferability. Extensive experiments demonstrate the efficacy of ReToMe-VA, particularly in surpassing state-of-the-art attacks in adversarial transferability by more than 14.16% on average.