Exploring Multimodal Prompts For Unsupervised Continuous Anomaly Detection

TL;DR

This paper introduces a multimodal prompt-based framework for unsupervised continuous anomaly detection that leverages visual and textual data to improve accuracy and robustness in complex scenes.

Contribution

It proposes a novel Continual Multimodal Prompt Memory Bank and a Defect-Semantic-Guided Adaptive Fusion Mechanism for enhanced anomaly detection.

Findings

Achieves state-of-the-art AUROC and AUPR on benchmark datasets.

Effectively captures normality in complex scenes using multimodal data.

Improves robustness against adversarial attacks.

Abstract

Unsupervised Continuous Anomaly Detection (UCAD) is gaining attention for effectively addressing the catastrophic forgetting and heavy computational burden issues in traditional Unsupervised Anomaly Detection (UAD). However, existing UCAD approaches that rely solely on visual information are insufficient to capture the manifold of normality in complex scenes, thereby impeding further gains in anomaly detection accuracy. To overcome this limitation, we propose an unsupervised continual anomaly detection framework grounded in multimodal prompting. Specifically, we introduce a Continual Multimodal Prompt Memory Bank (CMPMB) that progressively distills and retains prototypical normal patterns from both visual and textual domains across consecutive tasks, yielding a richer representation of normality. Furthermore, we devise a Defect-Semantic-Guided Adaptive Fusion Mechanism (DSG-AFM) that integrates an Adaptive Normalization Module (ANM) with a Dynamic Fusion Strategy (DFS) to jointly enhance detection accuracy and adversarial robustness. Benchmark experiments on MVTec AD and VisA datasets show that our approach achieves state-of-the-art (SOTA) performance on image-level AUROC and pixel-level AUPR metrics.