# Multi-Branch Network with Multi-Feature Enhancement for Improving the Generalization of Facial Forgery Detection

## TL;DR

This paper introduces a new deep learning model to detect fake faces, especially those created with advanced diffusion models, improving detection accuracy and generalization.

## Contribution

The novel M2EH model combines multi-branch networks and adaptive feature fusion to enhance generalization in facial forgery detection.

## Key findings

- M2EH outperforms existing methods on various deepfake datasets.
- The proposed model effectively integrates features from multiple branches to detect subtle traces of facial forgeries.
- A new dataset, HybridGenFace, is introduced to improve adaptability by including forgeries from both GANs and diffusion models.

## Abstract

The rapid development of deepfake facial technology has led to facial fraud, posing a significant threat to social security. With the advent of diffusion models, the realism of forged facial images has increased, making detection increasingly challenging. However, the existing detection methods primarily focus on identifying facial forgeries generated by generative adversarial networks; they may struggle to generalize when faced with novel forgery techniques like diffusion models. To address this challenge, a multi-branch network with multi-feature enhancement (M2EH) model for improving the generalization of facial forgery detection is proposed in this paper. First, a multi-branch network is constructed, wherein diverse features are extracted through the three parallel branches of the network, allowing for extensive analysis into the subtle traces of facial forgeries. Then, an adaptive feature concatenation mechanism is proposed to integrate the diverse features extracted from the three branches, obtaining the effective fused representation by optimizing the weights of each feature channel. To further enhance the facial forgery detection ability, spatial pyramid pooling is introduced into the classifier to augment the fused features. Finally, independent loss functions are designed for each branch to ensure the effective learning of specific features while promoting collaborative optimization of the model through the overall loss function. Additionally, to improve model adaptability, a large-scale deepfake facial dataset, HybridGenFace, is built, which includes counterfeit images generated by both generative adversarial networks and diffusion models, addressing the limitations of existing datasets concerning a single forgery type. Experimental results show that M2EH outperforms most of the existing methods on various deepfake datasets.