COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection

TL;DR

COMICS is an end-to-end multi-face forgery detection framework that leverages bi-grained contrastive learning to identify subtle forgery traces at multiple levels, outperforming existing methods especially in complex, real-world scenarios.

Contribution

The paper introduces COMICS, a novel end-to-end framework that combines face extraction with forgery detection using bi-grained contrastive learning for improved accuracy.

Findings

Outperforms existing methods on OpenForensics and FFIW datasets.

Effectively captures forgery traces at both coarse and fine levels.

Demonstrates robustness in complex, multi-face scenarios.

Abstract

DeepFakes have raised serious societal concerns, leading to a great surge in detection-based forensics methods in recent years. Face forgery recognition is a standard detection method that usually follows a two-phase pipeline. While those methods perform well in ideal experimental environment, they face challenges when dealing with DeepFakes in the wild involving complex background and multiple faces of varying sizes. Moreover, most face forgery recognition methods can only process one face at a time. One straightforward way to address this issue is to simultaneous process multi-face by integrating face extraction and forgery detection in an end-to-end fashion by adapting advanced object detection architectures. However, as these object detection architectures are designed to capture the discriminative features of different object categories rather than the subtle forgery traces among the faces, the direct adaptation suffers from limited representation ability. In this paper, we propose COMICS, an end-to-end framework for multi-face forgery detection. COMICS integrates face extraction and forgery detection in a seamless manner and adapts to advanced object detection architectures. The proposed bi-grained contrastive learning approach explores face forgery traces at both the coarse- and fine-grained levels. Specifically, coarse-grained level contrastive learning captures the discriminative features among positive and negative proposal pairs at multiple layers produced by the proposal generator, and fine-grained level contrastive learning captures the pixel-wise discrepancy between the forged and original areas of the same face and the pixel-wise content inconsistency among different faces. Extensive experiments on the OpenForensics and FFIW datasets demonstrate that our method outperforms other counterparts and shows great potential for being integrated into various architectures.