Celeb-DF++: A Large-scale Challenging Video DeepFake Benchmark for Generalizable Forensics

TL;DR

Celeb-DF++ is a large, diverse video DeepFake dataset designed to challenge and evaluate the generalizability of DeepFake detection methods across multiple forgery scenarios and techniques.

Contribution

The paper introduces Celeb-DF++, a comprehensive large-scale DeepFake dataset with diverse forgery types and evaluation protocols for assessing detection generalizability.

Findings

Existing detection methods show limited generalization on Celeb-DF++.

Celeb-DF++ covers three forgery scenarios with 22 different DeepFake methods.

The dataset exposes the limitations of current DeepFake detection techniques.

Abstract

The rapid advancement of AI technologies has significantly increased the diversity of DeepFake videos circulating online, posing a pressing challenge for \textit{generalizable forensics}, \ie, detecting a wide range of unseen DeepFake types using a single model. Addressing this challenge requires datasets that are not only large-scale but also rich in forgery diversity. However, most existing datasets, despite their scale, include only a limited variety of forgery types, making them insufficient for developing generalizable detection methods. Therefore, we build upon our earlier Celeb-DF dataset and introduce {Celeb-DF++}, a new large-scale and challenging video DeepFake benchmark dedicated to the generalizable forensics challenge. Celeb-DF++ covers three commonly encountered forgery scenarios: Face-swap (FS), Face-reenactment (FR), and Talking-face (TF). Each scenario contains a substantial number of high-quality forged videos, generated using a total of 22 various recent DeepFake methods. These methods differ in terms of architectures, generation pipelines, and targeted facial regions, covering the most prevalent DeepFake cases witnessed in the wild. We also introduce evaluation protocols for measuring the generalizability of 24 recent detection methods, highlighting the limitations of existing detection methods and the difficulty of our new dataset.