Rethinking Gradient Operator for Exposing AI-enabled Face Forgeries

TL;DR

This paper introduces two novel modules that incorporate gradient operators into CNNs to enhance the detection of face forgeries, outperforming previous methods across multiple datasets.

Contribution

The work proposes two plug-and-play modules, tensor pre-processing and manipulation trace attention, that leverage gradient operators to improve face forgery detection.

Findings

Proposed modules outperform prior methods on five datasets.

Tensor pre-processing improves accuracy by at least 4.60%.

Modules can be integrated into CNNs for end-to-end training.

Abstract

For image forensics, convolutional neural networks (CNNs) tend to learn content features rather than subtle manipulation traces, which limits forensic performance. Existing methods predominantly solve the above challenges by following a general pipeline, that is, subtracting the original pixel value from the predicted pixel value to make CNNs pay attention to the manipulation traces. However, due to the complicated learning mechanism, these methods may bring some unnecessary performance losses. In this work, we rethink the advantages of gradient operator in exposing face forgery, and design two plug-and-play modules by combining gradient operator with CNNs, namely tensor pre-processing (TP) and manipulation trace attention (MTA) module. Specifically, TP module refines the feature tensor of each channel in the network by gradient operator to highlight the manipulation traces and improve the feature representation. Moreover, MTA module considers two dimensions, namely channel and manipulation traces, to force the network to learn the distribution of manipulation traces. These two modules can be seamlessly integrated into CNNs for end-to-end training. Experiments show that the proposed network achieves better results than prior works on five public datasets. Especially, TP module greatly improves the accuracy by at least 4.60% compared with the existing pre-processing module only via simple tensor refinement. The code is available at: https://github.com/EricGzq/GocNet-pytorch.