Anomagic: Crossmodal Prompt-driven Zero-shot Anomaly Generation

TL;DR

Anomagic is a zero-shot anomaly generation method that uses crossmodal prompts to create realistic anomalies without exemplars, improving downstream detection accuracy and enabling versatile anomaly synthesis.

Contribution

It introduces a novel crossmodal prompt encoding scheme and a large dataset, AnomVerse, for training a versatile anomaly generation model.

Findings

Outperforms prior methods in realism and variety of generated anomalies

Enhances downstream anomaly detection accuracy

Can generate anomalies for any normal image using user prompts

Abstract

We propose Anomagic, a zero-shot anomaly generation method that produces semantically coherent anomalies without requiring any exemplar anomalies. By unifying both visual and textual cues through a crossmodal prompt encoding scheme, Anomagic leverages rich contextual information to steer an inpainting-based generation pipeline. A subsequent contrastive refinement strategy enforces precise alignment between synthesized anomalies and their masks, thereby bolstering downstream anomaly detection accuracy. To facilitate training, we introduce AnomVerse, a collection of 12,987 anomaly-mask-caption triplets assembled from 13 publicly available datasets, where captions are automatically generated by multimodal large language models using structured visual prompts and template-based textual hints. Extensive experiments demonstrate that Anomagic trained on AnomVerse can synthesize more realistic and varied anomalies than prior methods, yielding superior improvements in downstream anomaly detection. Furthermore, Anomagic can generate anomalies for any normal-category image using user-defined prompts, establishing a versatile foundation model for anomaly generation.