Two-in-one Knowledge Distillation for Efficient Facial Forgery Detection

TL;DR

This paper introduces a novel two-in-one knowledge distillation framework that effectively merges dual-branch spatial and frequency information into a compact single-branch model, significantly improving facial forgery detection efficiency.

Contribution

The paper proposes a new distillation method that successfully consolidates dual-branch spatial and frequency features into a single model, overcoming previous performance degradation issues.

Findings

Achieves superior detection accuracy on FaceForensics++ and Celeb-DF datasets.

Reduces model complexity with fewer parameters while maintaining high performance.

Demonstrates effective merging of dual-branch information into a single-branch network.

Abstract

Facial forgery detection is a crucial but extremely challenging topic, with the fast development of forgery techniques making the synthetic artefact highly indistinguishable. Prior works show that by mining both spatial and frequency information the forgery detection performance of deep learning models can be vastly improved. However, leveraging multiple types of information usually requires more than one branch in the neural network, which makes the model heavy and cumbersome. Knowledge distillation, as an important technique for efficient modelling, could be a possible remedy. We find that existing knowledge distillation methods have difficulties distilling a dual-branch model into a single-branch model. More specifically, knowledge distillation on both the spatial and frequency branches has degraded performance than distillation only on the spatial branch. To handle such problem, we propose a novel two-in-one knowledge distillation framework which can smoothly merge the information from a large dual-branch network into a small single-branch network, with the help of different dedicated feature projectors and the gradient homogenization technique. Experimental analysis on two datasets, FaceForensics++ and Celeb-DF, shows that our proposed framework achieves superior performance for facial forgery detection with much fewer parameters.