TIPS Over Tricks: Simple Prompts for Effective Zero-shot Anomaly Detection

TL;DR

This paper introduces TIPS, a vision-language model trained with spatially aware objectives, to improve zero-shot anomaly detection by addressing CLIP's limitations in localization and sensitivity, resulting in better performance across industrial datasets.

Contribution

The paper proposes TIPS, a backbone trained with spatially aware objectives, and a decoupled prompt approach for enhanced zero-shot anomaly detection without complex auxiliary modules.

Findings

Improves image-level detection accuracy by up to 3.9%.

Enhances pixel-level localization by up to 6.9%.

Demonstrates strong generalization across seven industrial datasets.

Abstract

Anomaly detection identifies departures from expected behavior in safety-critical settings. When target-domain normal data are unavailable, zero-shot anomaly detection (ZSAD) leverages vision-language models (VLMs). However, CLIP's coarse image-text alignment limits both localization and detection due to (i) spatial misalignment and (ii) weak sensitivity to fine-grained anomalies; prior works compensate with complex auxiliary modules yet largely overlook the choice of backbone. We revisit the backbone and use TIPS-a VLM trained with spatially aware objectives. While TIPS alleviates CLIP's issues, it exposes a distributional gap between global and local features. We address this with decoupled prompts-fixed for image-level detection and learnable for pixel-level localization-and by injecting local evidence into the global score. Without CLIP-specific tricks, our TIPS-based pipeline improves image-level performance by 1.1-3.9% and pixel-level by 1.5-6.9% across seven industrial datasets, delivering strong generalization with a lean architecture. Code is available at github.com/AlirezaSalehy/Tipsomaly.