Analyzing VLM-Based Approaches for Anomaly Classification and Segmentation

TL;DR

This paper provides a comprehensive analysis of Vision-Language Models like CLIP for anomaly detection, focusing on classification and segmentation, and evaluates various architectural strategies across multiple benchmarks.

Contribution

It offers a systematic investigation of VLM-based anomaly detection methods, highlighting their strengths, limitations, and practical insights for industrial applications.

Findings

VLMs enable effective zero-shot anomaly detection without extensive labeled data.

Different architectural paradigms impact accuracy and efficiency variably.

Cross-domain generalization remains a key challenge for VLM-based methods.

Abstract

Vision-Language Models (VLMs), particularly CLIP, have revolutionized anomaly detection by enabling zero-shot and few-shot defect identification without extensive labeled datasets. By learning aligned representations of images and text, VLMs facilitate anomaly classification and segmentation through natural language descriptions of normal and abnormal states, eliminating traditional requirements for task-specific training or defect examples. This project presents a comprehensive analysis of VLM-based approaches for anomaly classification (AC) and anomaly segmentation (AS). We systematically investigate key architectural paradigms including sliding window-based dense feature extraction (WinCLIP), multi-stage feature alignment with learnable projections (AprilLab framework), and compositional prompt ensemble strategies. Our analysis evaluates these methods across critical dimensions: feature extraction mechanisms, text-visual alignment strategies, prompt engineering techniques, zero-shot versus few-shot trade-offs, computational efficiency, and cross-domain generalization. Through rigorous experimentation on benchmarks such as MVTec AD and VisA, we compare classification accuracy, segmentation precision, and inference efficiency. The primary contribution is a foundational understanding of how and why VLMs succeed in anomaly detection, synthesizing practical insights for method selection and identifying current limitations. This work aims to facilitate informed adoption of VLM-based methods in industrial quality control and guide future research directions.