On the Holistic Approach for Detecting Human Image Forgery

TL;DR

This paper presents HuForDet, a comprehensive framework for detecting human image forgeries that combines face and full-body analysis using advanced neural modules and a new dataset, achieving state-of-the-art results.

Contribution

The paper introduces HuForDet, a novel holistic detection framework with dual-branch architecture and a new dataset, addressing the fragmentation in existing forgery detection methods.

Findings

HuForDet outperforms existing methods in forgery detection accuracy.

The dual-branch architecture effectively captures both facial and full-body inconsistencies.

The new HuFor dataset enhances training and evaluation of forgery detection models.

Abstract

The rapid advancement of AI-generated content (AIGC) has escalated the threat of deepfakes, from facial manipulations to the synthesis of entire photorealistic human bodies. However, existing detection methods remain fragmented, specializing either in facial-region forgeries or full-body synthetic images, and consequently fail to generalize across the full spectrum of human image manipulations. We introduce HuForDet, a holistic framework for human image forgery detection, which features a dual-branch architecture comprising: (1) a face forgery detection branch that employs heterogeneous experts operating in both RGB and frequency domains, including an adaptive Laplacian-of-Gaussian (LoG) module designed to capture artifacts ranging from fine-grained blending boundaries to coarse-scale texture irregularities; and (2) a contextualized forgery detection branch that leverages a Multi-Modal Large Language Model (MLLM) to analyze full-body semantic consistency, enhanced with a confidence estimation mechanism that dynamically weights its contribution during feature fusion. We curate a human image forgery (HuFor) dataset that unifies existing face forgery data with a new corpus of full-body synthetic humans. Extensive experiments show that our HuForDet achieves state-of-the-art forgery detection performance and superior robustness across diverse human image forgeries.