Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues

TL;DR

This paper introduces F3-Net, a frequency-aware face forgery detection model that leverages frequency clues to identify subtle manipulation artifacts, especially effective on compressed media.

Contribution

The paper presents a novel two-stream framework that integrates frequency-aware decomposed images and local frequency statistics for improved forgery detection.

Findings

F3-Net outperforms state-of-the-art methods across all compression qualities.

Significant accuracy gains on low-quality media.

Effective in detecting subtle forgery artifacts.

Abstract

As realistic facial manipulation technologies have achieved remarkable progress, social concerns about potential malicious abuse of these technologies bring out an emerging research topic of face forgery detection. However, it is extremely challenging since recent advances are able to forge faces beyond the perception ability of human eyes, especially in compressed images and videos. We find that mining forgery patterns with the awareness of frequency could be a cure, as frequency provides a complementary viewpoint where either subtle forgery artifacts or compression errors could be well described. To introduce frequency into the face forgery detection, we propose a novel Frequency in Face Forgery Network (F3-Net), taking advantages of two different but complementary frequency-aware clues, 1) frequency-aware decomposed image components, and 2) local frequency statistics, to deeply mine the forgery patterns via our two-stream collaborative learning framework. We apply DCT as the applied frequency-domain transformation. Through comprehensive studies, we show that the proposed F3-Net significantly outperforms competing state-of-the-art methods on all compression qualities in the challenging FaceForensics++ dataset, especially wins a big lead upon low-quality media.