Knowledge-Guided Prompt Learning for Deepfake Facial Image Detection

TL;DR

This paper introduces a knowledge-guided prompt learning approach that leverages large language models and test-time prompt tuning to improve deepfake facial image detection, especially under domain shift conditions.

Contribution

The paper presents a novel method combining language model prompts and test-time tuning to enhance deepfake detection accuracy and robustness.

Findings

Outperforms state-of-the-art methods on DeepFakeFaceForensics dataset

Effectively mitigates domain shift between training and testing data

Significantly improves detection performance in real-world scenarios

Abstract

Recent generative models demonstrate impressive performance on synthesizing photographic images, which makes humans hardly to distinguish them from pristine ones, especially on realistic-looking synthetic facial images. Previous works mostly focus on mining discriminative artifacts from vast amount of visual data. However, they usually lack the exploration of prior knowledge and rarely pay attention to the domain shift between training categories (e.g., natural and indoor objects) and testing ones (e.g., fine-grained human facial images), resulting in unsatisfactory detection performance. To address these issues, we propose a novel knowledge-guided prompt learning method for deepfake facial image detection. Specifically, we retrieve forgery-related prompts from large language models as expert knowledge to guide the optimization of learnable prompts. Besides, we elaborate test-time prompt tuning to alleviate the domain shift, achieving significant performance improvement and boosting the application in real-world scenarios. Extensive experiments on DeepFakeFaceForensics dataset show that our proposed approach notably outperforms state-of-the-art methods.