Detect and Locate: Exposing Face Manipulation by Semantic- and Noise-level Telltales

TL;DR

This paper introduces a novel method for detecting and localizing face manipulations by combining semantic segmentation and noise analysis, achieving state-of-the-art accuracy and localization performance.

Contribution

It presents a simple yet effective approach that integrates high-level semantic and low-level noise clues for face manipulation detection and localization.

Findings

Achieves state-of-the-art detection accuracy.

Provides remarkable localization of manipulated regions.

Combines semantic and noise-level features effectively.

Abstract

The technological advancements of deep learning have enabled sophisticated face manipulation schemes, raising severe trust issues and security concerns in modern society. Generally speaking, detecting manipulated faces and locating the potentially altered regions are challenging tasks. Herein, we propose a conceptually simple but effective method to efficiently detect forged faces in an image while simultaneously locating the manipulated regions. The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image. Furthermore, a noise map is estimated, playing a complementary role in capturing low-level clues and subsequently empowering decision-making. Finally, the features from these two modules are combined to distinguish fake faces. Extensive experiments show that the proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.