ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection

TL;DR

This paper introduces ForensicTransfer, a weakly-supervised domain adaptation method using an autoencoder-based forensic embedding to improve forgery detection on unseen manipulation methods with limited training data.

Contribution

The paper proposes a novel autoencoder-based forensic embedding that enhances transferability and detection accuracy for unseen image manipulations with minimal supervision.

Findings

Achieves up to 85% accuracy on unseen forgery methods.

Reaches around 95% accuracy with few training examples.

Significantly outperforms prior transferability methods.

Abstract

Distinguishing manipulated from real images is becoming increasingly difficult as new sophisticated image forgery approaches come out by the day. Naive classification approaches based on Convolutional Neural Networks (CNNs) show excellent performance in detecting image manipulations when they are trained on a specific forgery method. However, on examples from unseen manipulation approaches, their performance drops significantly. To address this limitation in transferability, we introduce Forensic-Transfer (FT). We devise a learning-based forensic detector which adapts well to new domains, i.e., novel manipulation methods and can handle scenarios where only a handful of fake examples are available during training. To this end, we learn a forensic embedding based on a novel autoencoder-based architecture that can be used to distinguish between real and fake imagery. The learned embedding acts as a form of anomaly detector; namely, an image manipulated from an unseen method will be detected as fake provided it maps sufficiently far away from the cluster of real images. Comparing to prior works, FT shows significant improvements in transferability, which we demonstrate in a series of experiments on cutting-edge benchmarks. For instance, on unseen examples, we achieve up to 85% in terms of accuracy, and with only a handful of seen examples, our performance already reaches around 95%.