GC-ConsFlow: Leveraging Optical Flow Residuals and Global Context for Robust Deepfake Detection

TL;DR

GC-ConsFlow is a dual-stream deepfake detection framework that combines spatial and temporal features, leveraging global context and optical flow residuals to improve robustness against natural facial motions and various compression scenarios.

Contribution

It introduces a novel dual-stream framework with global context aggregation and flow-gradient temporal consistency, enhancing detection of subtle artifacts and motion inconsistencies in deepfake videos.

Findings

Outperforms state-of-the-art methods in various compression scenarios

Effectively captures subtle spatial artifacts and temporal inconsistencies

Demonstrates robustness against natural facial motions

Abstract

The rapid development of Deepfake technology has enabled the generation of highly realistic manipulated videos, posing severe social and ethical challenges. Existing Deepfake detection methods primarily focused on either spatial or temporal inconsistencies, often neglecting the interplay between the two or suffering from interference caused by natural facial motions. To address these challenges, we propose the global context consistency flow (GC-ConsFlow), a novel dual-stream framework that effectively integrates spatial and temporal features for robust Deepfake detection. The global grouped context aggregation module (GGCA), integrated into the global context-aware frame flow stream (GCAF), enhances spatial feature extraction by aggregating grouped global context information, enabling the detection of subtle, spatial artifacts within frames. The flow-gradient temporal consistency stream (FGTC), rather than directly modeling the residuals, it is used to improve the robustness of temporal feature extraction against the inconsistency introduced by unnatural facial motion using optical flow residuals and gradient-based features. By combining these two streams, GC-ConsFlow demonstrates the effectiveness and robustness in capturing complementary spatiotemporal forgery traces. Extensive experiments show that GC-ConsFlow outperforms existing state-of-the-art methods in detecting Deepfake videos under various compression scenarios.