Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection

TL;DR

This paper investigates how up-sampling operations in CNN-based generators create generalized artifacts in synthetic images, proposing a novel method that improves deepfake detection accuracy by analyzing pixel relationships.

Contribution

It introduces the concept of Neighboring Pixel Relationships (NPR) to characterize artifacts from up-sampling, enhancing deepfake detection performance across diverse generative models.

Findings

NPR effectively captures structural artifacts from up-sampling.

The method achieves 11.6% improvement over existing deepfake detection techniques.

Analysis on 28 generative models demonstrates robustness and generalization.

Abstract

Recently, the proliferation of highly realistic synthetic images, facilitated through a variety of GANs and Diffusions, has significantly heightened the susceptibility to misuse. While the primary focus of deepfake detection has traditionally centered on the design of detection algorithms, an investigative inquiry into the generator architectures has remained conspicuously absent in recent years. This paper contributes to this lacuna by rethinking the architectures of CNN-based generators, thereby establishing a generalized representation of synthetic artifacts. Our findings illuminate that the up-sampling operator can, beyond frequency-based artifacts, produce generalized forgery artifacts. In particular, the local interdependence among image pixels caused by upsampling operators is significantly demonstrated in synthetic images generated by GAN or diffusion. Building upon this observation, we introduce the concept of Neighboring Pixel Relationships(NPR) as a means to capture and characterize the generalized structural artifacts stemming from up-sampling operations. A comprehensive analysis is conducted on an open-world dataset, comprising samples generated by \tft{28 distinct generative models}. This analysis culminates in the establishment of a novel state-of-the-art performance, showcasing a remarkable \tft{11.6\%} improvement over existing methods. The code is available at https://github.com/chuangchuangtan/NPR-DeepfakeDetection.