DiffusionFF: A Diffusion-based Framework for Joint Face Forgery Detection and Fine-Grained Artifact Localization

TL;DR

DiffusionFF is a novel diffusion-based framework that jointly detects face forgeries and localizes artifacts with high accuracy, improving explainability and trust in deepfake detection systems.

Contribution

It introduces a new encoder-decoder architecture combining a pretrained forgery detector with a diffusion model for detailed artifact localization.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Provides detailed localization maps that enhance model explainability.

Improves detection accuracy by fusing localization with semantic features.

Abstract

The rapid evolution of deepfake technologies demands robust and reliable face forgery detection algorithms. While determining whether an image has been manipulated remains essential, the ability to precisely localize forgery clues is also important for enhancing model explainability and building user trust. To address this dual challenge, we introduce DiffusionFF, a diffusion-based framework that simultaneously performs face forgery detection and fine-grained artifact localization. Our key idea is to establish a novel encoder-decoder architecture: a pretrained forgery detector serves as a powerful "artifact encoder", and a denoising diffusion model is repurposed as an "artifact decoder". Conditioned on multi-scale forgery-related features extracted by the encoder, the decoder progressively synthesizes a detailed artifact localization map. We then fuse this fine-grained localization map with high-level semantic features from the forgery detector, leading to substantial improvements in detection capability. Extensive experiments show that DiffusionFF achieves state-of-the-art (SOTA) performance across multiple benchmarks, underscoring its superior effectiveness and explainability.