Self-supervised Multi-Modal Video Forgery Attack Detection

TL;DR

This paper introduces a self-supervised multimodal approach combining vision and wireless signals to detect video forgery attacks, achieving high accuracy without external annotations.

Contribution

It proposes a novel self-supervised training strategy for multimodal forgery detection that adapts to different environments and achieves competitive accuracy.

Findings

Achieved 94.38% forgery attack detection accuracy.

Attained perfect human detection accuracy.

Method works without external annotations.

Abstract

Video forgery attack threatens the surveillance system by replacing the video captures with unrealistic synthesis, which can be powered by the latest augment reality and virtual reality technologies. From the machine perception aspect, visual objects often have RF signatures that are naturally synchronized with them during recording. In contrast to video captures, the RF signatures are more difficult to attack given their concealed and ubiquitous nature. In this work, we investigate multimodal video forgery attack detection methods using both vision and wireless modalities. Since wireless signal-based human perception is environmentally sensitive, we propose a self-supervised training strategy to enable the system to work without external annotation and thus can adapt to different environments. Our method achieves a perfect human detection accuracy and a high forgery attack detection accuracy of 94.38% which is comparable with supervised methods.