Adapted-MoE: Mixture of Experts with Test-Time Adaption for Anomaly Detection

TL;DR

This paper introduces Adapted-MoE, a novel approach for anomaly detection that uses a mixture of experts with test-time adaptation to handle distribution variations within normal samples, significantly improving detection performance.

Contribution

The paper proposes a mixture of experts model with a routing network and test-time adaptation to address distribution bias in anomaly detection tasks, which is a novel approach in this field.

Findings

Achieved 2.18%-7.20% increase in I-AUROC over baseline.

Achieved 1.57%-16.30% increase in P-AUROC over baseline.

Outperformed current state-of-the-art methods on Texture AD benchmark.

Abstract

Most unsupervised anomaly detection methods based on representations of normal samples to distinguish anomalies have recently made remarkable progress. However, existing methods only learn a single decision boundary for distinguishing the samples within the training dataset, neglecting the variation in feature distribution for normal samples even in the same category in the real world. Furthermore, it was not considered that a distribution bias still exists between the test set and the train set. Therefore, we propose an Adapted-MoE which contains a routing network and a series of expert models to handle multiple distributions of same-category samples by divide and conquer. Specifically, we propose a routing network based on representation learning to route same-category samples into the subclasses feature space. Then, a series of expert models are utilized to learn the representation of various normal samples and construct several independent decision boundaries. We propose the test-time adaption to eliminate the bias between the unseen test sample representation and the feature distribution learned by the expert model. Our experiments are conducted on a dataset that provides multiple subclasses from three categories, namely Texture AD benchmark. The Adapted-MoE significantly improves the performance of the baseline model, achieving 2.18%-7.20% and 1.57%-16.30% increase in I-AUROC and P-AUROC, which outperforms the current state-of-the-art methods. Our code is available at https://github.com/.