How Close are Other Computer Vision Tasks to Deepfake Detection?

TL;DR

This paper introduces a new measurement called 'model separability' to evaluate pre-trained models' effectiveness for deepfake detection, revealing that face recognition and self-supervised models are most promising.

Contribution

The paper presents a novel 'model separability' metric and systematically benchmarks various pre-trained models, highlighting the effectiveness of face recognition and self-supervised models for deepfake detection.

Findings

Face recognition models are more related to deepfake detection.

Self-supervised models outperform supervised models in separation tasks.

Fine-tuned self-supervised models achieve the best detection results, with some overfitting risk.

Abstract

In this paper, we challenge the conventional belief that supervised ImageNet-trained models have strong generalizability and are suitable for use as feature extractors in deepfake detection. We present a new measurement, "model separability," for visually and quantitatively assessing a model's raw capacity to separate data in an unsupervised manner. We also present a systematic benchmark for determining the correlation between deepfake detection and other computer vision tasks using pre-trained models. Our analysis shows that pre-trained face recognition models are more closely related to deepfake detection than other models. Additionally, models trained using self-supervised methods are more effective in separation than those trained using supervised methods. After fine-tuning all models on a small deepfake dataset, we found that self-supervised models deliver the best results, but there is a risk of overfitting. Our results provide valuable insights that should help researchers and practitioners develop more effective deepfake detection models.