OmniAD: Detect and Understand Industrial Anomaly via Multimodal Reasoning

TL;DR

OmniAD is a multimodal framework that combines visual and textual reasoning to detect and analyze industrial anomalies with high accuracy, surpassing existing models on multiple benchmarks.

Contribution

The paper introduces OmniAD, a novel multimodal reasoning framework that unifies anomaly detection and understanding, utilizing visual and textual processes for detailed industrial analysis.

Findings

Achieves 79.1 on MMAD benchmark, outperforming prior models.

Effectively integrates visual perception with reasoning for anomaly analysis.

Demonstrates strong performance across multiple benchmarks.

Abstract

While anomaly detection has made significant progress, generating detailed analyses that incorporate industrial knowledge remains a challenge. To address this gap, we introduce OmniAD, a novel framework that unifies anomaly detection and understanding for fine-grained analysis. OmniAD is a multimodal reasoner that combines visual and textual reasoning processes. The visual reasoning provides detailed inspection by leveraging Text-as-Mask Encoding to perform anomaly detection through text generation without manually selected thresholds. Following this, Visual Guided Textual Reasoning conducts comprehensive analysis by integrating visual perception. To enhance few-shot generalization, we employ an integrated training strategy that combines supervised fine-tuning (SFT) with reinforcement learning (GRPO), incorporating three sophisticated reward functions. Experimental results demonstrate that OmniAD achieves a performance of 79.1 on the MMAD benchmark, surpassing models such as Qwen2.5-VL-7B and GPT-4o. It also shows strong results across multiple anomaly detection benchmarks. These results highlight the importance of enhancing visual perception for effective reasoning in anomaly understanding. All codes and models will be publicly available.