Fine-grained Abnormality Prompt Learning for Zero-shot Anomaly Detection
Jiawen Zhu, Yew-Soon Ong, Chunhua Shen, Guansong Pang
TL;DR
FAPrompt introduces a novel framework with fine-grained abnormality prompts and adaptive modules, significantly improving zero-shot anomaly detection across diverse datasets and anomaly types.
Contribution
The paper proposes FAPrompt, a new approach that learns decomposed abnormality prompts and data-dependent priors for enhanced zero-shot anomaly detection.
Findings
Outperforms state-of-the-art methods on 19 datasets
Effective in both industrial defect and medical anomaly detection
Improves accuracy at image and pixel levels
Abstract
Current zero-shot anomaly detection (ZSAD) methods show remarkable success in prompting large pre-trained vision-language models to detect anomalies in a target dataset without using any dataset-specific training or demonstration. However, these methods often focus on crafting/learning prompts that capture only coarse-grained semantics of abnormality, e.g., high-level semantics like "damaged", "imperfect", or "defective" objects. They therefore have limited capability in recognizing diverse abnormality details that deviate from these general abnormal patterns in various ways. To address this limitation, we propose FAPrompt, a novel framework designed to learn Fine-grained Abnormality Prompts for accurate ZSAD. To this end, a novel Compound Abnormality Prompt learning (CAP) module is introduced in FAPrompt to learn a set of complementary, decomposed abnormality prompts, where abnormality…
Peer Reviews
Decision·Submitted to ICLR 2025
1. The paper is clearly written and well-organized, making complex ideas accessible. Diagrams and figures effectively illustrate key ideas, enhancing reader comprehension. 2. The experiments are thorough, encompassing 19 diverse datasets from both industrial and medical domains. The results demonstrate substantial improvements over current state-of-the-art methods, reflecting the high quality of the proposed approach.
1. There is marginal improvement in pixel-level ZSAD in Table 2. In contrast, the simple AnomalyCLIP achieves comparable, and even superior, results on industrial datasets. 2. The design of DAP is very similar to CoCoOp, and using a fixed M across images with varying scales of anomalous regions is unreasonable. 3. Missing necessary baseline: AnomalyCLIP with an ensemble of multiple abnormality prompts with orthogonal constraint loss; otherwise, it is difficult to justify the fine-grained prompts
The motivation of this study is reasonable, and the proposed method is novel. Additionally, the method has been validated across diverse datasets.
1. The description of the learnable layers in the DAP module is missing. 2. The visualization in Figure 1 of the proposed methodology conflicts with the actual experimental results shown.
1. FAPrompt addresses a key limitation in current ZSAD methods, which often struggle to identify fine-grained, specific abnormalities. By introducing learnable fine-grained abnormality prompts, FAPrompt improves both the accuracy and applicability of ZSAD in diverse contexts. 2. Comprehensive experiments show that FAPrompt outperforms state-of-the-art methods by notable margins (3%-5% improvements in AUC/AP) across both image- and pixel-level ZSAD tasks.
1. The novelty of FAPrompt is limited. The design of CAP seems a simple combination of prompt tuning and prototype learning. 2. Besides CoOp and CoCoOp, other state-of-the-art prompt tuning approaches are expected for comparisons (e.g. PromptSRC[1] and TCP[2]). *Reference* [1] Khattak, Muhammad Uzair, et al. "Self-regulating prompts: Foundational model adaptation without forgetting." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. [2] Yao, Hantao, Rui Zhang, and
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAnomaly Detection Techniques and Applications
MethodsSparse Evolutionary Training