FB-CLIP: Fine-Grained Zero-Shot Anomaly Detection with Foreground-Background Disentanglement

TL;DR

FB-CLIP is a novel framework that improves zero-shot fine-grained anomaly detection by disentangling foreground and background features and enhancing semantic representations in vision-language models.

Contribution

It introduces multi-strategy textual representations and visual separation techniques to address foreground-background entanglement in CLIP for anomaly detection.

Findings

Achieves accurate zero-shot anomaly detection and localization.

Effectively distinguishes anomalies from complex backgrounds.

Outperforms existing methods in fine-grained anomaly detection tasks.

Abstract

Fine-grained anomaly detection is crucial in industrial and medical applications, but labeled anomalies are often scarce, making zero-shot detection challenging. While vision-language models like CLIP offer promising solutions, they struggle with foreground-background feature entanglement and coarse textual semantics. We propose FB-CLIP, a framework that enhances anomaly localization via multi-strategy textual representations and foreground-background separation. In the textual modality, it combines End-of-Text features, global-pooled representations, and attention-weighted token features for richer semantic cues. In the visual modality, multi-view soft separation along identity, semantic, and spatial dimensions, together with background suppression, reduces interference and improves discriminability. Semantic Consistency Regularization (SCR) aligns image features with normal and abnormal textual prototypes, suppressing uncertain matches and enlarging semantic gaps. Experiments show that FB-CLIP effectively distinguishes anomalies from complex backgrounds, achieving accurate fine-grained anomaly detection and localization under zero-shot settings.