Temporal Shuffling for Defending Deep Action Recognition Models against Adversarial Attacks

TL;DR

This paper introduces a novel defense method using temporal shuffling to protect 3D CNN-based video action recognition models from adversarial attacks, exploiting their robustness to frame order randomization.

Contribution

It is the first to propose a training-free defense mechanism based on temporal shuffling for video action recognition models against adversarial attacks.

Findings

Models are less reliant on frame order than expected.

Motion monotonicity persists after randomization, aiding robustness.

Adversarial perturbations are sensitive to temporal destruction.

Abstract

Recently, video-based action recognition methods using convolutional neural networks (CNNs) achieve remarkable recognition performance. However, there is still lack of understanding about the generalization mechanism of action recognition models. In this paper, we suggest that action recognition models rely on the motion information less than expected, and thus they are robust to randomization of frame orders. Furthermore, we find that motion monotonicity remaining after randomization also contributes to such robustness. Based on this observation, we develop a novel defense method using temporal shuffling of input videos against adversarial attacks for action recognition models. Another observation enabling our defense method is that adversarial perturbations on videos are sensitive to temporal destruction. To the best of our knowledge, this is the first attempt to design a defense method without additional training for 3D CNN-based video action recognition models.