Adversarially Robust Video Perception by Seeing Motion

TL;DR

This paper introduces a method to enhance the adversarial robustness of video perception models by restoring motion information, leveraging the intrinsic structure of video data to defend against attacks without requiring human annotations.

Contribution

The authors propose a novel inference-time technique that restores perceived motion to improve robustness of video models against adversarial attacks, even under adaptive attack scenarios.

Findings

Restoring motion improves adversarial robustness in video models.

The method remains effective against adaptive attacks.

Empirical results on UCF-101 and HMDB-51 datasets support the approach.

Abstract

Despite their excellent performance, state-of-the-art computer vision models often fail when they encounter adversarial examples. Video perception models tend to be more fragile under attacks, because the adversary has more places to manipulate in high-dimensional data. In this paper, we find one reason for video models' vulnerability is that they fail to perceive the correct motion under adversarial perturbations. Inspired by the extensive evidence that motion is a key factor for the human visual system, we propose to correct what the model sees by restoring the perceived motion information. Since motion information is an intrinsic structure of the video data, recovering motion signals can be done at inference time without any human annotation, which allows the model to adapt to unforeseen, worst-case inputs. Visualizations and empirical experiments on UCF-101 and HMDB-51 datasets show that restoring motion information in deep vision models improves adversarial robustness. Even under adaptive attacks where the adversary knows our defense, our algorithm is still effective. Our work provides new insight into robust video perception algorithms by using intrinsic structures from the data. Our webpage is available at https://motion4robust.cs.columbia.edu.