Trident: Detecting Face Forgeries with Adversarial Triplet Learning

TL;DR

Trident is a novel face forgery detection framework that uses triplet learning and adversarial training to improve robustness against unseen deepfake techniques, outperforming existing models.

Contribution

Introduces Trident, a face forgery detection method combining triplet learning with domain-adversarial training for enhanced generalization.

Findings

Effective in detecting diverse forgeries

Robust against unseen manipulation techniques

Outperforms existing detection models

Abstract

As face forgeries generated by deep neural networks become increasingly sophisticated, detecting face manipulations in digital media has posed a significant challenge, underscoring the importance of maintaining digital media integrity and combating visual disinformation. Current detection models, predominantly based on supervised training with domain-specific data, often falter against forgeries generated by unencountered techniques. In response to this challenge, we introduce \textit{Trident}, a face forgery detection framework that employs triplet learning with a Siamese network architecture for enhanced adaptability across diverse forgery methods. \textit{Trident} is trained on curated triplets to isolate nuanced differences of forgeries, capturing fine-grained features that distinguish pristine samples from manipulated ones while controlling for other variables. To further enhance generalizability, we incorporate domain-adversarial training with a forgery discriminator. This adversarial component guides our embedding model towards forgery-agnostic representations, improving its robustness to unseen manipulations. In addition, we prevent gradient flow from the classifier head to the embedding model, avoiding overfitting induced by artifacts peculiar to certain forgeries. Comprehensive evaluations across multiple benchmarks and ablation studies demonstrate the effectiveness of our framework. We will release our code in a GitHub repository.