One Language-Free Foundation Model Is Enough for Universal Vision Anomaly Detection

TL;DR

This paper introduces a simple, universal framework for visual anomaly detection that eliminates the need for language encoders and complex adaptation, achieving state-of-the-art results across diverse benchmarks.

Contribution

It proposes a decoupled, parameter-efficient approach that removes reliance on language models for anomaly detection, simplifying and improving the process.

Findings

Outperforms state-of-the-art zero-/few-shot methods

Surpasses full-shot anomaly detection performance

Highly parameter-efficient with only 0.002M learnable parameters

Abstract

Universal visual anomaly detection (AD) aims to identify anomaly images and segment anomaly regions towards open and dynamic scenarios, following zero- and few-shot paradigms without any dataset-specific fine-tuning. We have witnessed significant progress in widely use of visual-language foundational models in recent approaches. However, current methods often struggle with complex prompt engineering, elaborate adaptation modules, and challenging training strategies, ultimately limiting their flexibility and generality. To address these issues, this paper rethinks the fundamental mechanism behind visual-language models for AD and presents an embarrassingly simple, general, and effective framework for Universal vision Anomaly Detection (UniADet). Specifically, we first find language encoder is used to derive decision weights for anomaly classification and segmentation, and then demonstrate that it is unnecessary for universal AD. Second, we propose an embarrassingly simple method to completely decouple classification and segmentation, and decouple cross-level features, i.e., learning independent weights for different tasks and hierarchical features. UniADet is highly simple (learning only decoupled weights), parameter-efficient (only 0.002M learnable parameters), general (adapting a variety of foundation models), and effective (surpassing state-of-the-art zero-/few-shot by a large margin and even full-shot AD methods for the first time) on 14 real-world AD benchmarks covering both industrial and medical domains. We will make the code and model of UniADet available at https://github.com/gaobb/UniADet.