Contrastive Pseudo Learning for Open-World DeepFake Attribution

TL;DR

This paper introduces a new benchmark and a novel contrastive pseudo learning framework for open-world deepfake attribution, addressing challenges in identifying various forgery types and unknown attacks.

Contribution

It proposes the Contrastive Pseudo Learning framework with a Global-Local Voting module and confidence-based pseudo-labeling, extending to a multi-stage paradigm for improved deepfake attribution.

Findings

Outperforms existing methods on the OW-DFA benchmark

Enhances interpretability of deepfake attribution

Improves security in deepfake detection

Abstract

The challenge in sourcing attribution for forgery faces has gained widespread attention due to the rapid development of generative techniques. While many recent works have taken essential steps on GAN-generated faces, more threatening attacks related to identity swapping or expression transferring are still overlooked. And the forgery traces hidden in unknown attacks from the open-world unlabeled faces still remain under-explored. To push the related frontier research, we introduce a new benchmark called Open-World DeepFake Attribution (OW-DFA), which aims to evaluate attribution performance against various types of fake faces under open-world scenarios. Meanwhile, we propose a novel framework named Contrastive Pseudo Learning (CPL) for the OW-DFA task through 1) introducing a Global-Local Voting module to guide the feature alignment of forged faces with different manipulated regions, 2) designing a Confidence-based Soft Pseudo-label strategy to mitigate the pseudo-noise caused by similar methods in unlabeled set. In addition, we extend the CPL framework with a multi-stage paradigm that leverages pre-train technique and iterative learning to further enhance traceability performance. Extensive experiments verify the superiority of our proposed method on the OW-DFA and also demonstrate the interpretability of deepfake attribution task and its impact on improving the security of deepfake detection area.