Training-Free Anomaly Generation via Dual-Attention Enhancement in Diffusion Model

TL;DR

This paper introduces AAG, a training-free framework leveraging Stable Diffusion with cross- and self-attention enhancements to generate realistic anomalies for industrial defect detection, improving data diversity and detection performance.

Contribution

AAG is a novel training-free anomaly generation method that enhances Stable Diffusion with attention mechanisms to produce high-fidelity anomalies without extra training data.

Findings

AAG effectively generates realistic anomalies in specific regions.

Generated anomalies improve downstream defect detection accuracy.

AAG outperforms existing methods in anomaly realism and utility.

Abstract

Industrial anomaly detection (AD) plays a significant role in manufacturing where a long-standing challenge is data scarcity. A growing body of works have emerged to address insufficient anomaly data via anomaly generation. However, these anomaly generation methods suffer from lack of fidelity or need to be trained with extra data. To this end, we propose a training-free anomaly generation framework dubbed AAG, which is based on Stable Diffusion (SD)'s strong generation ability for effective anomaly image generation. Given a normal image, mask and a simple text prompt, AAG can generate realistic and natural anomalies in the specific regions and simultaneously keep contents in other regions unchanged. In particular, we propose Cross-Attention Enhancement (CAE) to re-engineer the cross-attention mechanism within Stable Diffusion based on the given mask. CAE increases the similarity between visual tokens in specific regions and text embeddings, which guides these generated visual tokens in accordance with the text description. Besides, generated anomalies need to be more natural and plausible with object in given image. We propose Self-Attention Enhancement (SAE) which improves similarity between each normal visual token and anomaly visual tokens. SAE ensures that generated anomalies are coherent with original pattern. Extensive experiments on MVTec AD and VisA datasets demonstrate effectiveness of AAG in anomaly generation and its utility. Furthermore, anomaly images generated by AAG can bolster performance of various downstream anomaly inspection tasks.