Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline

TL;DR

This paper introduces a new task and dataset for generating detailed attribution reports for manipulated facial images, combining localization and natural language explanations to improve understanding of forgeries.

Contribution

It presents a novel multimodal task, a large-scale dataset (MMTT), and a unified framework (ForgeryTalker) for explainable facial forgery attribution.

Findings

ForgeryTalker achieves 59.3 CIDEr score in report generation.

The dataset contains 152,217 samples with precise annotations.

The model performs well on both localization and explanation tasks.

Abstract

Existing facial forgery detection methods typically focus on binary classification or pixel-level localization, providing little semantic insight into the nature of the manipulation. To address this, we introduce Forgery Attribution Report Generation, a new multimodal task that jointly localizes forged regions ("Where") and generates natural language explanations grounded in the editing process ("Why"). This dual-focus approach goes beyond traditional forensics, providing a comprehensive understanding of the manipulation. To enable research in this domain, we present Multi-Modal Tamper Tracing (MMTT), a large-scale dataset of 152,217 samples, each with a process-derived ground-truth mask and a human-authored textual description, ensuring high annotation precision and linguistic richness. We further propose ForgeryTalker, a unified end-to-end framework that integrates vision and language via a shared encoder (image encoder + Q-former) and dual decoders for mask and text generation, enabling coherent cross-modal reasoning. Experiments show that ForgeryTalker achieves competitive performance on both report generation and forgery localization subtasks, i.e., 59.3 CIDEr and 73.67 IoU, respectively, establishing a baseline for explainable multimedia forensics. Dataset and code will be released to foster future research.