Towards Fine-Grained Vision-Language Alignment for Few-Shot Anomaly Detection

TL;DR

This paper introduces a multi-level fine-grained semantic captioning approach to improve anomaly localization in few-shot settings by better aligning textual descriptions with visual regions.

Contribution

It proposes the MFSC method for automatic multi-level textual descriptions and the FineGrainedAD framework with MLLP and MLSA components for enhanced anomaly detection.

Findings

Achieves superior performance on MVTec-AD and VisA datasets.

Improves semantic alignment between text and visual regions.

Enhances localization accuracy in few-shot anomaly detection.

Abstract

Few-shot anomaly detection (FSAD) methods identify anomalous regions with few known normal samples. Most existing methods rely on the generalization ability of pre-trained vision-language models (VLMs) to recognize potentially anomalous regions through feature similarity between text descriptions and images. However, due to the lack of detailed textual descriptions, these methods can only pre-define image-level descriptions to match each visual patch token to identify potential anomalous regions, which leads to the semantic misalignment between image descriptions and patch-level visual anomalies, achieving sub-optimal localization performance. To address the above issues, we propose the Multi-Level Fine-Grained Semantic Caption (MFSC) to provide multi-level and fine-grained textual descriptions for existing anomaly detection datasets with automatic construction pipeline. Based on the MFSC, we propose a novel framework named FineGrainedAD to improve anomaly localization performance, which consists of two components: Multi-Level Learnable Prompt (MLLP) and Multi-Level Semantic Alignment (MLSA). MLLP introduces fine-grained semantics into multi-level learnable prompts through automatic replacement and concatenation mechanism, while MLSA designs region aggregation strategy and multi-level alignment training to facilitate learnable prompts better align with corresponding visual regions. Experiments demonstrate that the proposed FineGrainedAD achieves superior overall performance in few-shot settings on MVTec-AD and VisA datasets.