Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in VIS and NIR Scenario

TL;DR

This paper introduces a novel spatial-temporal frequency analysis method using DCT for detecting forged videos, effectively capturing both spatial and temporal clues in VIS and NIR scenarios.

Contribution

It proposes FCAN-DCT, a new network that leverages frequency domain features across space and time for improved video forgery detection.

Findings

Effective in VIS and NIR scenarios

Outperforms existing methods on benchmark datasets

First to use NIR modality for video forgery detection

Abstract

In recent years, with the rapid development of face editing and generation, more and more fake videos are circulating on social media, which has caused extreme public concerns. Existing face forgery detection methods based on frequency domain find that the GAN forged images have obvious grid-like visual artifacts in the frequency spectrum compared to the real images. But for synthesized videos, these methods only confine to single frame and pay little attention to the most discriminative part and temporal frequency clue among different frames. To take full advantage of the rich information in video sequences, this paper performs video forgery detection on both spatial and temporal frequency domains and proposes a Discrete Cosine Transform-based Forgery Clue Augmentation Network (FCAN-DCT) to achieve a more comprehensive spatial-temporal feature representation. FCAN-DCT consists of a backbone network and two branches: Compact Feature Extraction (CFE) module and Frequency Temporal Attention (FTA) module. We conduct thorough experimental assessments on two visible light (VIS) based datasets WildDeepfake and Celeb-DF (v2), and our self-built video forgery dataset DeepfakeNIR, which is the first video forgery dataset on near-infrared modality. The experimental results demonstrate the effectiveness of our method on detecting forgery videos in both VIS and NIR scenarios.