Self-Supervised Graph Transformer for Deepfake Detection

TL;DR

This paper introduces a self-supervised graph Transformer-based framework for deepfake detection that exhibits strong generalization across datasets, manipulations, and common video perturbations, with explainability features.

Contribution

The study presents a novel self-supervised pre-training approach combined with graph Transformer architecture for improved deepfake detection and interpretability.

Findings

Outperforms state-of-the-art methods in cross-dataset tests.

Demonstrates robustness against video compression and blur.

Provides explainability through relevancy maps.

Abstract

Deepfake detection methods have shown promising results in recognizing forgeries within a given dataset, where training and testing take place on the in-distribution dataset. However, their performance deteriorates significantly when presented with unseen samples. As a result, a reliable deepfake detection system must remain impartial to forgery types, appearance, and quality for guaranteed generalizable detection performance. Despite various attempts to enhance cross-dataset generalization, the problem remains challenging, particularly when testing against common post-processing perturbations, such as video compression or blur. Hence, this study introduces a deepfake detection framework, leveraging a self-supervised pre-training model that delivers exceptional generalization ability, withstanding common corruptions and enabling feature explainability. The framework comprises three key components: a feature extractor based on vision Transformer architecture that is pre-trained via self-supervised contrastive learning methodology, a graph convolution network coupled with a Transformer discriminator, and a graph Transformer relevancy map that provides a better understanding of manipulated regions and further explains the model's decision. To assess the effectiveness of the proposed framework, several challenging experiments are conducted, including in-data distribution performance, cross-dataset, cross-manipulation generalization, and robustness against common post-production perturbations. The results achieved demonstrate the remarkable effectiveness of the proposed deepfake detection framework, surpassing the current state-of-the-art approaches.