A Novel Framework for Assessment of Learning-based Detectors in Realistic Conditions with Application to Deepfake Detection

TL;DR

This paper introduces a comprehensive framework for evaluating learning-based detectors, like deepfake detectors, under realistic conditions involving common distortions, and demonstrates how data augmentation improves their robustness.

Contribution

It presents a new assessment framework for detectors in realistic scenarios and proposes a data augmentation strategy to enhance detector generalization.

Findings

Assessment framework reveals performance drops under distortions

Data augmentation improves detector robustness significantly

Framework applicable to various detection tasks

Abstract

Deep convolutional neural networks have shown remarkable results on multiple detection tasks. Despite the significant progress, the performance of such detectors are often assessed in public benchmarks under non-realistic conditions. Specifically, impact of conventional distortions and processing operations such as compression, noise, and enhancement are not sufficiently studied. This paper proposes a rigorous framework to assess performance of learning-based detectors in more realistic situations. An illustrative example is shown under deepfake detection context. Inspired by the assessment results, a data augmentation strategy based on natural image degradation process is designed, which significantly improves the generalization ability of two deepfake detectors.