PromptMAD: Cross-Modal Prompting for Multi-Class Visual Anomaly Localization

TL;DR

PromptMAD introduces a cross-modal prompting framework that leverages vision-language alignment and semantic guidance to enhance unsupervised multi-class visual anomaly detection and localization, achieving state-of-the-art results.

Contribution

It proposes a novel cross-modal prompting approach combining CLIP-based semantic guidance, a specialized segmentor, and Focal loss to improve anomaly detection across diverse categories.

Findings

Achieves 98.35% mean AUC on MVTec-AD

Attains 66.54% AP, outperforming previous methods

Demonstrates robustness across multiple object categories

Abstract

Visual anomaly detection in multi-class settings poses significant challenges due to the diversity of object categories, the scarcity of anomalous examples, and the presence of camouflaged defects. In this paper, we propose PromptMAD, a cross-modal prompting framework for unsupervised visual anomaly detection and localization that integrates semantic guidance through vision-language alignment. By leveraging CLIP-encoded text prompts describing both normal and anomalous class-specific characteristics, our method enriches visual reconstruction with semantic context, improving the detection of subtle and textural anomalies. To further address the challenge of class imbalance at the pixel level, we incorporate Focal loss function, which emphasizes hard-to-detect anomalous regions during training. Our architecture also includes a supervised segmentor that fuses multi-scale convolutional features with Transformer-based spatial attention and diffusion iterative refinement, yielding precise and high-resolution anomaly maps. Extensive experiments on the MVTec-AD dataset demonstrate that our method achieves state-of-the-art pixel-level performance, improving mean AUC to 98.35% and AP to 66.54%, while maintaining efficiency across diverse categories.