ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels

TL;DR

ADMoE introduces a novel mixture-of-experts framework that effectively learns from noisy labels for anomaly detection, significantly improving performance over existing methods across multiple datasets.

Contribution

This work presents the first anomaly detection framework that leverages noisy labels using a mixture-of-experts architecture, enabling scalable and specialized learning from multiple noisy sources.

Findings

Up to 34% performance improvement over baseline methods.

Outperforms 13 leading anomaly detection baselines.

Effective on diverse datasets including enterprise security data.

Abstract

Existing works on anomaly detection (AD) rely on clean labels from human annotators that are expensive to acquire in practice. In this work, we propose a method to leverage weak/noisy labels (e.g., risk scores generated by machine rules for detecting malware) that are cheaper to obtain for anomaly detection. Specifically, we propose ADMoE, the first framework for anomaly detection algorithms to learn from noisy labels. In a nutshell, ADMoE leverages mixture-of-experts (MoE) architecture to encourage specialized and scalable learning from multiple noisy sources. It captures the similarities among noisy labels by sharing most model parameters, while encouraging specialization by building "expert" sub-networks. To further juice out the signals from noisy labels, ADMoE uses them as input features to facilitate expert learning. Extensive results on eight datasets (including a proprietary enterprise security dataset) demonstrate the effectiveness of ADMoE, where it brings up to 34% performance improvement over not using it. Also, it outperforms a total of 13 leading baselines with equivalent network parameters and FLOPS. Notably, ADMoE is model-agnostic to enable any neural network-based detection methods to handle noisy labels, where we showcase its results on both multiple-layer perceptron (MLP) and the leading AD method DeepSAD.