OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wild

TL;DR

OpenForensics introduces a large-scale, challenging dataset with detailed annotations for multi-face forgery detection and segmentation in natural scenes, advancing research in deepfake countermeasures.

Contribution

The paper presents the first large-scale dataset with rich face-wise annotations for multi-face forgery detection and segmentation in-the-wild, along with benchmark evaluations of state-of-the-art methods.

Findings

State-of-the-art detection methods perform variably on the dataset.

The dataset reveals significant challenges in multi-face forgery localization.

Benchmark results highlight areas for future improvement.

Abstract

The proliferation of deepfake media is raising concerns among the public and relevant authorities. It has become essential to develop countermeasures against forged faces in social media. This paper presents a comprehensive study on two new countermeasure tasks: multi-face forgery detection and segmentation in-the-wild. Localizing forged faces among multiple human faces in unrestricted natural scenes is far more challenging than the traditional deepfake recognition task. To promote these new tasks, we have created the first large-scale dataset posing a high level of challenges that is designed with face-wise rich annotations explicitly for face forgery detection and segmentation, namely OpenForensics. With its rich annotations, our OpenForensics dataset has great potentials for research in both deepfake prevention and general human face detection. We have also developed a suite of benchmarks for these tasks by conducting an extensive evaluation of state-of-the-art instance detection and segmentation methods on our newly constructed dataset in various scenarios. The dataset, benchmark results, codes, and supplementary materials will be publicly available on our project page: https://sites.google.com/view/ltnghia/research/openforensics