Finding Facial Forgery Artifacts with Parts-Based Detectors

TL;DR

This paper introduces parts-based detectors for facial forgery detection that are generalizable, explainable, and effective across multiple datasets, providing insights into manipulated video artifacts.

Contribution

The paper presents a novel parts-based detection approach that enhances generalization and interpretability in deepfake video detection.

Findings

Detectors effectively generalize across datasets

Provides explainability by identifying facial regions used for detection

Analyzes dataset-specific artifacts and statistics

Abstract

Manipulated videos, especially those where the identity of an individual has been modified using deep neural networks, are becoming an increasingly relevant threat in the modern day. In this paper, we seek to develop a generalizable, explainable solution to detecting these manipulated videos. To achieve this, we design a series of forgery detection systems that each focus on one individual part of the face. These parts-based detection systems, which can be combined and used together in a single architecture, meet all of our desired criteria - they generalize effectively between datasets and give us valuable insights into what the network is looking at when making its decision. We thus use these detectors to perform detailed empirical analysis on the FaceForensics++, Celeb-DF, and Facebook Deepfake Detection Challenge datasets, examining not just what the detectors find but also collecting and analyzing useful related statistics on the datasets themselves.