Do Deepfake Detectors Work in Reality?

TL;DR

This paper reveals that common post-processing techniques like super-resolution significantly reduce the effectiveness of deepfake detectors in real-world scenarios, highlighting a critical gap between research and practical application.

Contribution

The study introduces the first real-world faceswap dataset and demonstrates how post-processing undermines current deepfake detection methods.

Findings

Deepfake detectors perform near random on real-world data.

Super-resolution post-processing significantly degrades detection accuracy.

A new dataset enables realistic evaluation of detection methods.

Abstract

Deepfakes, particularly those involving faceswap-based manipulations, have sparked significant societal concern due to their increasing realism and potential for misuse. Despite rapid advancements in generative models, detection methods have not kept pace, creating a critical gap in defense strategies. This disparity is further amplified by the disconnect between academic research and real-world applications, which often prioritize different objectives and evaluation criteria. In this study, we take a pivotal step toward bridging this gap by presenting a novel observation: the post-processing step of super-resolution, commonly employed in real-world scenarios, substantially undermines the effectiveness of existing deepfake detection methods. To substantiate this claim, we introduce and publish the first real-world faceswap dataset, collected from popular online faceswap platforms. We then qualitatively evaluate the performance of state-of-the-art deepfake detectors on real-world deepfakes, revealing that their accuracy approaches the level of random guessing. Furthermore, we quantitatively demonstrate the significant performance degradation caused by common post-processing techniques. By addressing this overlooked challenge, our study underscores a critical avenue for enhancing the robustness and practical applicability of deepfake detection methods in real-world settings.