Just One Moment: Structural Vulnerability of Deep Action Recognition against One Frame Attack

TL;DR

This paper reveals that deep learning-based action recognition models are highly vulnerable to a novel one frame adversarial attack, which can fool models with minimal and inconspicuous perturbations on a single video frame.

Contribution

The study introduces the one frame attack for action recognition models and demonstrates their structural vulnerability and the effectiveness of universal perturbations.

Findings

High fooling rates achieved with one frame attack

Models are structurally susceptible to minimal perturbations

Universal perturbations effective across scenarios

Abstract

The video-based action recognition task has been extensively studied in recent years. In this paper, we study the structural vulnerability of deep learning-based action recognition models against the adversarial attack using the one frame attack that adds an inconspicuous perturbation to only a single frame of a given video clip. Our analysis shows that the models are highly vulnerable against the one frame attack due to their structural properties. Experiments demonstrate high fooling rates and inconspicuous characteristics of the attack. Furthermore, we show that strong universal one frame perturbations can be obtained under various scenarios. Our work raises the serious issue of adversarial vulnerability of the state-of-the-art action recognition models in various perspectives.