Learning to mask: Towards generalized face forgery detection

TL;DR

This paper proposes a novel face forgery detection method that enhances generalization to unseen forgeries by reducing overfitting on specific features through attention-guided training and feature mixup strategies.

Contribution

It introduces a teacher-student framework with attention guidance and feature mixup to improve forgery detection generalization without relying on data augmentation.

Findings

Achieves promising results on unseen forgeries

Performs well on highly compressed data

Reduces overfitting to specific forgery types

Abstract

Generalizability to unseen forgery types is crucial for face forgery detectors. Recent works have made significant progress in terms of generalization by synthetic forgery data augmentation. In this work, we explore another path for improving the generalization. Our goal is to reduce the features that are easy to learn in the training phase, so as to reduce the risk of overfitting on specific forgery types. Specifically, in our method, a teacher network takes as input the face images and generates an attention map of the deep features by a diverse multihead attention ViT. The attention map is used to guide a student network to focus on the low-attended features by reducing the highly-attended deep features. A deep feature mixup strategy is also proposed to synthesize forgeries in the feature domain. Experiments demonstrate that, without data augmentation, our method is able to achieve promising performances on unseen forgeries and highly compressed data.