Exploring Specular Reflection Inconsistency for Generalizable Face Forgery Detection

TL;DR

This paper introduces a novel face forgery detection method that leverages inconsistencies in specular reflection, a complex physical attribute, to identify AI-generated deepfakes with high accuracy, especially those created by diffusion models.

Contribution

The paper proposes a new detection approach based on specular reflection inconsistency, utilizing a Retinex-based texture estimation and a cross-attention network to improve forgery detection robustness.

Findings

Achieves superior detection performance on multiple deepfake datasets.

Effectively detects diffusion-generated forgeries with high accuracy.

Demonstrates the importance of physical attribute inconsistencies in forgery detection.

Abstract

Detecting deepfakes has become increasingly challenging as forgery faces synthesized by AI-generated methods, particularly diffusion models, achieve unprecedented quality and resolution. Existing forgery detection approaches relying on spatial and frequency features demonstrate limited efficacy against high-quality, entirely synthesized forgeries. In this paper, we propose a novel detection method grounded in the observation that facial attributes governed by complex physical laws and multiple parameters are inherently difficult to replicate. Specifically, we focus on illumination, particularly the specular reflection component in the Phong illumination model, which poses the greatest replication challenge due to its parametric complexity and nonlinear formulation. We introduce a fast and accurate face texture estimation method based on Retinex theory to enable precise specular reflection separation. Furthermore, drawing from the mathematical formulation of specular reflection, we posit that forgery evidence manifests not only in the specular reflection itself but also in its relationship with corresponding face texture and direct light. To address this issue, we design the Specular-Reflection-Inconsistency-Network (SRI-Net), incorporating a two-stage cross-attention mechanism to capture these correlations and integrate specular reflection related features with image features for robust forgery detection. Experimental results demonstrate that our method achieves superior performance on both traditional deepfake datasets and generative deepfake datasets, particularly those containing diffusion-generated forgery faces.