Complement Face Forensic Detection and Localization with FacialLandmarks

TL;DR

This paper introduces a new face forensic localization dataset with diverse manipulated images and proposes a dual-branch method incorporating facial landmarks, achieving superior detection and localization performance over existing techniques.

Contribution

The paper presents the first comprehensive face forensic localization dataset and a novel dual-branch method that leverages facial landmarks for improved detection and localization accuracy.

Findings

Facial landmarks enhance forensic detection performance.

The proposed method outperforms state-of-the-art on multiple datasets.

Effective in low-quality video scenarios.

Abstract

Recently, Generative Adversarial Networks (GANs) and image manipulating methods are becoming more powerful and can produce highly realistic face images beyond human recognition which have raised significant concerns regarding the authenticity of digital media. Although there have been some prior works that tackle face forensic classification problem, it is not trivial to estimate edited locations from classification predictions. In this paper, we propose, to the best of our knowledge, the first rigorous face forensic localization dataset, which consists of genuine, generated, and manipulated face images. In particular, the pristine parts contain face images from CelebA and FFHQ datasets. The fake images are generated from various GANs methods, namely DCGANs, LSGANs, BEGANs, WGAN-GP, ProGANs, and StyleGANs. Lastly, the edited subset is generated from StarGAN and SEFCGAN based on free-form masks. In total, the dataset contains about 1.3 million facial images labelled with corresponding binary masks. Based on the proposed dataset, we demonstrated that explicit adding facial landmarks information in addition to input images improves the performance. In addition, our proposed method consists of two branches and can coherently predict face forensic detection and localization to outperform the previous state-of-the-art techniques on the newly proposed dataset as well as the faceforecsic++ dataset especially on low-quality videos.