Noise-Informed Diffusion-Generated Image Detection with Anomaly Attention

TL;DR

This paper introduces NASA-Swin, a novel diffusion-generated image detection method leveraging noise patterns and a self-attention module, achieving state-of-the-art results especially on unseen diffusion models.

Contribution

It proposes a noise-focused self-attention module integrated into Swin Transformer and a cross-modality fusion technique for improved detection of diffusion-generated images.

Findings

Effective detection of diffusion images using noise patterns.

State-of-the-art performance on unseen diffusion models.

Enhanced feature learning through cross-modality fusion.

Abstract

With the rapid development of image generation technologies, especially the advancement of Diffusion Models, the quality of synthesized images has significantly improved, raising concerns among researchers about information security. To mitigate the malicious abuse of diffusion models, diffusion-generated image detection has proven to be an effective countermeasure.However, a key challenge for forgery detection is generalising to diffusion models not seen during training. In this paper, we address this problem by focusing on image noise. We observe that images from different diffusion models share similar noise patterns, distinct from genuine images. Building upon this insight, we introduce a novel Noise-Aware Self-Attention (NASA) module that focuses on noise regions to capture anomalous patterns. To implement a SOTA detection model, we incorporate NASA into Swin Transformer, forming an novel detection architecture NASA-Swin. Additionally, we employ a cross-modality fusion embedding to combine RGB and noise images, along with a channel mask strategy to enhance feature learning from both modalities. Extensive experiments demonstrate the effectiveness of our approach in enhancing detection capabilities for diffusion-generated images. When encountering unseen generation methods, our approach achieves the state-of-the-art performance.Our code is available at https://github.com/WeinanGuan/NASA-Swin.