Multi-class Image Anomaly Detection for Practical Applications: Requirements and Robust Solutions

TL;DR

This paper introduces HierCore, a hierarchical coreset framework for multi-class image anomaly detection that performs robustly across various scenarios with or without class labels, addressing a key challenge in practical applications.

Contribution

The paper formalizes requirements for multi-class anomaly detection models and proposes HierCore, a novel, label-agnostic framework that consistently satisfies these criteria across different training and testing conditions.

Findings

HierCore outperforms existing methods in all tested scenarios.

HierCore maintains stable detection accuracy regardless of label availability.

Existing methods are less robust across different label conditions.

Abstract

Recent advances in image anomaly detection have extended unsupervised learning-based models from single-class settings to multi-class frameworks, aiming to improve efficiency in training time and model storage. When a single model is trained to handle multiple classes, it often underperforms compared to class-specific models in terms of per-class detection accuracy. Accordingly, previous studies have primarily focused on narrowing this performance gap. However, the way class information is used, or not used, remains a relatively understudied factor that could influence how detection thresholds are defined in multi-class image anomaly detection. These thresholds, whether class-specific or class-agnostic, significantly affect detection outcomes. In this study, we identify and formalize the requirements that a multi-class image anomaly detection model must satisfy under different conditions, depending on whether class labels are available during training and evaluation. We then re-examine existing methods under these criteria. To meet these challenges, we propose Hierarchical Coreset (HierCore), a novel framework designed to satisfy all defined requirements. HierCore operates effectively even without class labels, leveraging a hierarchical memory bank to estimate class-wise decision criteria for anomaly detection. We empirically validate the applicability and robustness of existing methods and HierCore under four distinct scenarios, determined by the presence or absence of class labels in the training and evaluation phases. The experimental results demonstrate that HierCore consistently meets all requirements and maintains strong, stable performance across all settings, highlighting its practical potential for real-world multi-class anomaly detection tasks.