Mover: Mask and Recovery based Facial Part Consistency Aware Method for Deepfake Video Detection

TL;DR

Mover is a novel deepfake detection method that exploits facial part inconsistencies by masking and recovering facial regions, making it difficult for fake faces to be accurately reconstructed while real faces are easily recovered.

Contribution

The paper introduces Mover, a deepfake detection approach that leverages unspecific facial part inconsistencies through masking and recovery, enhancing detection robustness against improved forgery techniques.

Findings

High detection accuracy on standard benchmarks

Effective in identifying inconsistencies in fake videos

Robust against various deepfake manipulation methods

Abstract

Deepfake techniques have been widely used for malicious purposes, prompting extensive research interest in developing Deepfake detection methods. Deepfake manipulations typically involve tampering with facial parts, which can result in inconsistencies across different parts of the face. For instance, Deepfake techniques may change smiling lips to an upset lip, while the eyes remain smiling. Existing detection methods depend on specific indicators of forgery, which tend to disappear as the forgery patterns are improved. To address the limitation, we propose Mover, a new Deepfake detection model that exploits unspecific facial part inconsistencies, which are inevitable weaknesses of Deepfake videos. Mover randomly masks regions of interest (ROIs) and recovers faces to learn unspecific features, which makes it difficult for fake faces to be recovered, while real faces can be easily recovered. Specifically, given a real face image, we first pretrain a masked autoencoder to learn facial part consistency by dividing faces into three parts and randomly masking ROIs, which are then recovered based on the unmasked facial parts. Furthermore, to maximize the discrepancy between real and fake videos, we propose a novel model with dual networks that utilize the pretrained encoder and masked autoencoder, respectively. 1) The pretrained encoder is finetuned for capturing the encoding of inconsistent information in the given video. 2) The pretrained masked autoencoder is utilized for mapping faces and distinguishing real and fake videos. Our extensive experiments on standard benchmarks demonstrate that Mover is highly effective.