Zero-Shot Anomaly Detection with Dual-Branch Prompt Selection

TL;DR

This paper presents PILOT, a zero-shot anomaly detection framework that uses dual-branch prompt learning and test-time adaptation to effectively detect and localize anomalies across unseen categories and domain shifts.

Contribution

The paper introduces a novel dual-branch prompt learning mechanism and a label-free test-time adaptation strategy for improved zero-shot anomaly detection.

Findings

Achieves state-of-the-art results on 13 benchmarks.

Effectively handles domain shifts in anomaly detection.

Improves both detection and localization performance.

Abstract

Zero-shot anomaly detection (ZSAD) enables identifying and localizing defects in unseen categories by relying solely on generalizable features rather than requiring any labeled examples of anomalies. However, existing ZSAD methods, whether using fixed or learned prompts, struggle under domain shifts because their training data are derived from limited training domains and fail to generalize to new distributions. In this paper, we introduce PILOT, a framework designed to overcome these challenges through two key innovations: (1) a novel dual-branch prompt learning mechanism that dynamically integrates a pool of learnable prompts with structured semantic attributes, enabling the model to adaptively weight the most relevant anomaly cues for each input image; and (2) a label-free test-time adaptation strategy that updates the learnable prompt parameters using high-confidence pseudo-labels from unlabeled test data. Extensive experiments on 13 industrial and medical benchmarks demonstrate that PILOT achieves state-of-the-art performance in both anomaly detection and localization under domain shift.