Conditioned Prompt-Optimization for Continual Deepfake Detection

TL;DR

This paper presents Prompt2Guard, a novel continual deepfake detection method using vision-language models and domain-specific prompts, achieving state-of-the-art results and improved efficiency in detecting evolving deepfake content.

Contribution

Introduction of Prompt2Guard, a prompt-based continual deepfake detection approach that enhances accuracy and efficiency without multiple forward passes, leveraging read-only prompts and text-prompt conditioning.

Findings

Achieved state-of-the-art performance on CDDB-Hard benchmark.

Demonstrated improved efficiency over previous VLM-based methods.

Effectively addresses challenges of continual deepfake detection.

Abstract

The rapid advancement of generative models has significantly enhanced the realism and customization of digital content creation. The increasing power of these tools, coupled with their ease of access, fuels the creation of photorealistic fake content, termed deepfakes, that raises substantial concerns about their potential misuse. In response, there has been notable progress in developing detection mechanisms to identify content produced by these advanced systems. However, existing methods often struggle to adapt to the continuously evolving landscape of deepfake generation. This paper introduces Prompt2Guard, a novel solution for exemplar-free continual deepfake detection of images, that leverages Vision-Language Models (VLMs) and domain-specific multimodal prompts. Compared to previous VLM-based approaches that are either bounded by prompt selection accuracy or necessitate multiple forward passes, we leverage a prediction ensembling technique with read-only prompts. Read-only prompts do not interact with VLMs internal representation, mitigating the need for multiple forward passes. Thus, we enhance efficiency and accuracy in detecting generated content. Additionally, our method exploits a text-prompt conditioning tailored to deepfake detection, which we demonstrate is beneficial in our setting. We evaluate Prompt2Guard on CDDB-Hard, a continual deepfake detection benchmark composed of five deepfake detection datasets spanning multiple domains and generators, achieving a new state-of-the-art. Additionally, our results underscore the effectiveness of our approach in addressing the challenges posed by continual deepfake detection, paving the way for more robust and adaptable solutions in deepfake detection.