Protecting Celebrities from DeepFake with Identity Consistency Transformer
Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Ting Zhang and, Weiming Zhang, Nenghai Yu, Dong Chen, Fang Wen, Baining Guo

TL;DR
This paper introduces the Identity Consistency Transformer, a face forgery detection method that leverages identity information and high-level semantics to identify deepfakes, especially effective on celebrity images and robust across datasets and image degradations.
Contribution
The paper presents a novel transformer-based approach that uses identity consistency loss for improved deepfake detection, with enhanced generalization and adaptability to real-world conditions.
Findings
Exhibits superior generalization across datasets and image degradations
Effective in detecting deepfakes involving celebrities
Can incorporate additional identity information for improved accuracy
Abstract
In this work we propose Identity Consistency Transformer, a novel face forgery detection method that focuses on high-level semantics, specifically identity information, and detecting a suspect face by finding identity inconsistency in inner and outer face regions. The Identity Consistency Transformer incorporates a consistency loss for identity consistency determination. We show that Identity Consistency Transformer exhibits superior generalization ability not only across different datasets but also across various types of image degradation forms found in real-world applications including deepfake videos. The Identity Consistency Transformer can be easily enhanced with additional identity information when such information is available, and for this reason it is especially well-suited for detecting face forgeries involving celebrities. Code will be released at…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Media Forensic Detection · Face recognition and analysis · Generative Adversarial Networks and Image Synthesis
MethodsMulti-Head Attention · Attention Is All You Need · Linear Layer · Dense Connections · Softmax · Absolute Position Encodings · Byte Pair Encoding · Position-Wise Feed-Forward Layer · Residual Connection · Layer Normalization
