Generalizable Face Forgery Detection via Separable Prompt Learning

TL;DR

This paper introduces Separable Prompt Learning (SePL), a novel method leveraging CLIP's text modality to improve face forgery detection, achieving strong generalization across datasets and methods.

Contribution

It proposes a new SePL strategy that disentangles forgery-related and irrelevant information using cross-modality alignment, enhancing detection performance.

Findings

Achieves superior cross-dataset detection accuracy.

Demonstrates strong generalization to unseen forgery methods.

Outperforms existing methods in various evaluation settings.

Abstract

Detecting face forgeries using CLIP has recently emerged as a promising and increasingly popular research direction. Owing to its rich visual knowledge acquired through large-scale pretraining, most existing methods typically rely on the visual encoder of CLIP, while paying limited attention to the text modality. Given the instructive nature of the text modality, we posit that it can be leveraged to instruct Deepfake detection with meticulous design. Accordingly, we shift the focus from the visual modality to the text modality and propose a new Separable Prompt Learning strategy (SePL) that enables CLIP to serve as an effective face forgery detector. The core idea of SePL is to disentangle forgery-specific and forgery-irrelevant information in images via two types of prompt learning, with the former enhancing detection. To achieve this disentangle, we describe a cross-modality alignment strategy and a set of dedicated objectives. Extensive experiments demonstrate that, with this simple adaptation, our method achieves competitive and even superior performance compared to other methods under both cross-dataset and cross-method evaluation, highlighting its strong generalizability. The codes have been released at https://github.com/OUC-YER/SePL-DeepfakeDetection