Fusion-SSAT: Unleashing the Potential of Self-supervised Auxiliary Task by Feature Fusion for Generalized Deepfake Detection

TL;DR

This paper introduces Fusion-SSAT, a method that combines self-supervised auxiliary tasks with feature fusion to improve the generalization and performance of deepfake detection across multiple datasets.

Contribution

It proposes a novel feature fusion approach that leverages self-supervised auxiliary tasks to enhance deepfake detection accuracy and generalizability.

Findings

Fusion-SSAT achieves superior cross-dataset performance.

Feature fusion from self-supervised tasks improves primary task accuracy.

Method outperforms current state-of-the-art detectors.

Abstract

In this work, we attempted to unleash the potential of self-supervised learning as an auxiliary task that can optimise the primary task of generalised deepfake detection. To explore this, we examined different combinations of the training schemes for these tasks that can be most effective. Our findings reveal that fusing the feature representation from self-supervised auxiliary tasks is a powerful feature representation for the problem at hand. Such a representation can leverage the ultimate potential and bring in a unique representation of both the self-supervised and primary tasks, achieving better performance for the primary task. We experimented on a large set of datasets, which includes DF40, FaceForensics++, Celeb-DF, DFD, FaceShifter, UADFV, and our results showed better generalizability on cross-dataset evaluation when compared with current state-of-the-art detectors.