AnomalyMoE: Towards a Language-free Generalist Model for Unified Visual Anomaly Detection

TL;DR

AnomalyMoE introduces a universal, hierarchical anomaly detection framework utilizing a Mixture-of-Experts architecture to effectively identify diverse anomalies across multiple domains without relying on language.

Contribution

The paper presents a novel hierarchical MoE-based model with expert diversity modules for unified anomaly detection across various modalities.

Findings

Achieves state-of-the-art performance on 8 diverse datasets.

Effectively detects anomalies at local, component, and global levels.

Outperforms specialized methods in their respective domains.

Abstract

Anomaly detection is a critical task across numerous domains and modalities, yet existing methods are often highly specialized, limiting their generalizability. These specialized models, tailored for specific anomaly types like textural defects or logical errors, typically exhibit limited performance when deployed outside their designated contexts. To overcome this limitation, we propose AnomalyMoE, a novel and universal anomaly detection framework based on a Mixture-of-Experts (MoE) architecture. Our key insight is to decompose the complex anomaly detection problem into three distinct semantic hierarchies: local structural anomalies, component-level semantic anomalies, and global logical anomalies. AnomalyMoE correspondingly employs three dedicated expert networks at the patch, component, and global levels, and is specialized in reconstructing features and identifying deviations at its designated semantic level. This hierarchical design allows a single model to concurrently understand and detect a wide spectrum of anomalies. Furthermore, we introduce an Expert Information Repulsion (EIR) module to promote expert diversity and an Expert Selection Balancing (ESB) module to ensure the comprehensive utilization of all experts. Experiments on 8 challenging datasets spanning industrial imaging, 3D point clouds, medical imaging, video surveillance, and logical anomaly detection demonstrate that AnomalyMoE establishes new state-of-the-art performance, significantly outperforming specialized methods in their respective domains.