Facial Forgery-based Deepfake Detection using Fine-Grained Features

TL;DR

This paper introduces a fine-grained classification approach for deepfake detection that emphasizes learning subtle, discriminative features to improve cross-dataset and cross-manipulation generalization, addressing limitations of existing CNN-based methods.

Contribution

It formulates deepfake detection as a fine-grained classification problem and proposes a novel method that suppresses background noise and learns multi-scale discriminative features.

Findings

Outperforms existing methods in cross-dataset scenarios

Achieves higher generalization across different manipulation techniques

Demonstrates robustness in various experimental settings

Abstract

Facial forgery by deepfakes has caused major security risks and raised severe societal concerns. As a countermeasure, a number of deepfake detection methods have been proposed. Most of them model deepfake detection as a binary classification problem using a backbone convolutional neural network (CNN) architecture pretrained for the task. These CNN-based methods have demonstrated very high efficacy in deepfake detection with the Area under the Curve (AUC) as high as $0.99$. However, the performance of these methods degrades significantly when evaluated across datasets and deepfake manipulation techniques. This draws our attention towards learning more subtle, local, and discriminative features for deepfake detection. In this paper, we formulate deepfake detection as a fine-grained classification problem and propose a new fine-grained solution to it. Specifically, our method is based on learning subtle and generalizable features by effectively suppressing background noise and learning discriminative features at various scales for deepfake detection. Through extensive experimental validation, we demonstrate the superiority of our method over the published research in cross-dataset and cross-manipulation generalization of deepfake detectors for the majority of the experimental scenarios.