FakeLocator: Robust Localization of GAN-Based Face Manipulations

TL;DR

FakeLocator is a novel method that leverages GAN architecture imperfections to accurately detect and localize fake regions in manipulated facial images, achieving high resolution and robustness across various methods and real-world degradations.

Contribution

This work introduces FakeLocator, the first approach using a gray-scale fakeness map for precise GAN-based face forgery localization, enhanced by attention mechanisms and data augmentation for universality.

Findings

Outperforms baselines on FaceForensics++ and DFFD datasets

Effective across seven GAN-based face generation methods

Robust against JPEG compression, noise, and blur

Abstract

Full face synthesis and partial face manipulation by virtue of the generative adversarial networks (GANs) and its variants have raised wide public concerns. In the multi-media forensics area, detecting and ultimately locating the image forgery has become an imperative task. In this work, we investigate the architecture of existing GAN-based face manipulation methods and observe that the imperfection of upsampling methods therewithin could be served as an important asset for GAN-synthesized fake image detection and forgery localization. Based on this basic observation, we have proposed a novel approach, termed FakeLocator, to obtain high localization accuracy, at full resolution, on manipulated facial images. To the best of our knowledge, this is the very first attempt to solve the GAN-based fake localization problem with a gray-scale fakeness map that preserves more information of fake regions. To improve the universality of FakeLocator across multifarious facial attributes, we introduce an attention mechanism to guide the training of the model. To improve the universality of FakeLocator across different DeepFake methods, we propose partial data augmentation and single sample clustering on the training images. Experimental results on popular FaceForensics++, DFFD datasets and seven different state-of-the-art GAN-based face generation methods have shown the effectiveness of our method. Compared with the baselines, our method performs better on various metrics. Moreover, the proposed method is robust against various real-world facial image degradations such as JPEG compression, low-resolution, noise, and blur.