ED$^4$: Explicit Data-level Debiasing for Deepfake Detection

TL;DR

ED$^4$ introduces a data-level debiasing framework for deepfake detection, addressing spatial and content biases explicitly to improve generalization across diverse datasets.

Contribution

The paper presents ED$^4$, a novel unified framework with ClockMix and AdvSCM modules for explicit data-level bias mitigation in deepfake detection.

Findings

Significantly improves detection accuracy across multiple datasets.

Effectively reduces spatial and content biases in deepfake detectors.

Enhances generalization ability of various deepfake detection models.

Abstract

Learning intrinsic bias from limited data has been considered the main reason for the failure of deepfake detection with generalizability. Apart from the discovered content and specific-forgery bias, we reveal a novel spatial bias, where detectors inertly anticipate observing structural forgery clues appearing at the image center, also can lead to the poor generalization of existing methods. We present ED$^4$, a simple and effective strategy, to address aforementioned biases explicitly at the data level in a unified framework rather than implicit disentanglement via network design. In particular, we develop ClockMix to produce facial structure preserved mixtures with arbitrary samples, which allows the detector to learn from an exponentially extended data distribution with much more diverse identities, backgrounds, local manipulation traces, and the co-occurrence of multiple forgery artifacts. We further propose the Adversarial Spatial Consistency Module (AdvSCM) to prevent extracting features with spatial bias, which adversarially generates spatial-inconsistent images and constrains their extracted feature to be consistent. As a model-agnostic debiasing strategy, ED$^4$ is plug-and-play: it can be integrated with various deepfake detectors to obtain significant benefits. We conduct extensive experiments to demonstrate its effectiveness and superiority over existing deepfake detection approaches.