AnoStyler: Text-Driven Localized Anomaly Generation via Lightweight Style Transfer

TL;DR

AnoStyler introduces a lightweight, text-guided style transfer method for realistic, localized anomaly generation from normal images, improving data diversity and detection performance.

Contribution

It presents a novel zero-shot anomaly generation approach using lightweight models and CLIP-based loss, overcoming realism, data, and model size limitations of prior methods.

Findings

Outperforms existing methods in image quality and diversity

Enhances anomaly detection accuracy with generated data

Operates efficiently with minimal data and model size

Abstract

Anomaly generation has been widely explored to address the scarcity of anomaly images in real-world data. However, existing methods typically suffer from at least one of the following limitations, hindering their practical deployment: (1) lack of visual realism in generated anomalies; (2) dependence on large amounts of real images; and (3) use of memory-intensive, heavyweight model architectures. To overcome these limitations, we propose AnoStyler, a lightweight yet effective method that frames zero-shot anomaly generation as text-guided style transfer. Given a single normal image along with its category label and expected defect type, an anomaly mask indicating the localized anomaly regions and two-class text prompts representing the normal and anomaly states are generated using generalizable category-agnostic procedures. A lightweight U-Net model trained with CLIP-based loss functions is used to stylize the normal image into a visually realistic anomaly image, where anomalies are localized by the anomaly mask and semantically aligned with the text prompts. Extensive experiments on the MVTec-AD and VisA datasets show that AnoStyler outperforms existing anomaly generation methods in generating high-quality and diverse anomaly images. Furthermore, using these generated anomalies helps enhance anomaly detection performance.