Face Forgery Detection by 3D Decomposition

TL;DR

This paper introduces a face forgery detection method based on 3D decomposition, highlighting subtle forgery cues in facial detail, and achieves state-of-the-art results by leveraging disentangled 3D shape, texture, and lighting components.

Contribution

The paper proposes a novel 3D decomposition approach to extract facial details for improved forgery detection, emphasizing the importance of direct light and identity texture.

Findings

Facial detail features improve detection accuracy.

The method outperforms existing state-of-the-art techniques.

Disentangling 3D components reveals subtle forgery cues.

Abstract

Detecting digital face manipulation has attracted extensive attention due to fake media's potential harms to the public. However, recent advances have been able to reduce the forgery signals to a low magnitude. Decomposition, which reversibly decomposes an image into several constituent elements, is a promising way to highlight the hidden forgery details. In this paper, we consider a face image as the production of the intervention of the underlying 3D geometry and the lighting environment, and decompose it in a computer graphics view. Specifically, by disentangling the face image into 3D shape, common texture, identity texture, ambient light, and direct light, we find the devil lies in the direct light and the identity texture. Based on this observation, we propose to utilize facial detail, which is the combination of direct light and identity texture, as the clue to detect the subtle forgery patterns. Besides, we highlight the manipulated region with a supervised attention mechanism and introduce a two-stream structure to exploit both face image and facial detail together as a multi-modality task. Extensive experiments indicate the effectiveness of the extra features extracted from the facial detail, and our method achieves the state-of-the-art performance.