Fine-Grained DINO Tuning with Dual Supervision for Face Forgery Detection

TL;DR

This paper introduces a lightweight, multi-task fine-tuning method for DINOv2 that improves face forgery detection and manipulation classification by leveraging artifact-specific cues with minimal additional parameters.

Contribution

It proposes the DFF-Adapter, a parameter-efficient multi-task fine-tuning approach that enhances forgery detection and manipulation classification using shared artifact cues.

Findings

Achieves state-of-the-art detection accuracy with only 3.5M trainable parameters.

Effectively classifies different deepfake manipulation methods.

Enhances artifact sensitivity through multi-task cooperative optimization.

Abstract

The proliferation of sophisticated deepfakes poses significant threats to information integrity. While DINOv2 shows promise for detection, existing fine-tuning approaches treat it as generic binary classification, overlooking distinct artifacts inherent to different deepfake methods. To address this, we propose a DeepFake Fine-Grained Adapter (DFF-Adapter) for DINOv2. Our method incorporates lightweight multi-head LoRA modules into every transformer block, enabling efficient backbone adaptation. DFF-Adapter simultaneously addresses authenticity detection and fine-grained manipulation type classification, where classifying forgery methods enhances artifact sensitivity. We introduce a shared branch propagating fine-grained manipulation cues to the authenticity head. This enables multi-task cooperative optimization, explicitly enhancing authenticity discrimination with manipulation-specific knowledge. Utilizing only 3.5M trainable parameters, our parameter-efficient approach achieves detection accuracy comparable to or even surpassing that of current complex state-of-the-art methods.