Generalize Your Face Forgery Detectors: An Insertable Adaptation Module Is All You Need

TL;DR

This paper introduces a plug-and-play insertable adaptation module that enhances face forgery detectors' ability to generalize to unseen forgeries using only unlabeled test data, without altering the original architecture.

Contribution

The authors propose a novel adaptation module with a class prototype-based classifier and feature calibrator, improving generalization of face forgery detectors in real-world scenarios.

Findings

Outperforms state-of-the-art methods in generalization across datasets.

Functions as a versatile plug-and-play component for various detectors.

Enhances detection accuracy with minimal additional training.

Abstract

A plethora of face forgery detectors exist to tackle facial deepfake risks. However, their practical application is hindered by the challenge of generalizing to forgeries unseen during the training stage. To this end, we introduce an insertable adaptation module that can adapt a trained off-the-shelf detector using only online unlabeled test data, without requiring modifications to the architecture or training process. Specifically, we first present a learnable class prototype-based classifier that generates predictions from the revised features and prototypes, enabling effective handling of various forgery clues and domain gaps during online testing. Additionally, we propose a nearest feature calibrator to further improve prediction accuracy and reduce the impact of noisy pseudo-labels during self-training. Experiments across multiple datasets show that our module achieves superior generalization compared to state-of-the-art methods. Moreover, it functions as a plug-and-play component that can be combined with various detectors to enhance the overall performance.