Uncertainty-aware Evaluation of Auxiliary Anomalies with the Expected Anomaly Posterior

TL;DR

This paper introduces the Expected Anomaly Posterior (EAP), an uncertainty-based metric for evaluating the quality of synthetic auxiliary anomalies in anomaly detection, improving model training and detection performance.

Contribution

The paper proposes EAP, a novel uncertainty-based score that quantifies auxiliary anomaly quality, addressing the lack of quality assessment methods for synthetic anomalies.

Findings

EAP outperforms 12 existing data quality estimators on 40 benchmark datasets.

EAP effectively measures the uncertainty of auxiliary anomalies, enhancing anomaly detection.

Synthetic anomalies of higher quality, as measured by EAP, lead to better detection performance.

Abstract

Anomaly detection is the task of identifying examples that do not behave as expected. Because anomalies are rare and unexpected events, collecting real anomalous examples is often challenging in several applications. In addition, learning an anomaly detector with limited (or no) anomalies often yields poor prediction performance. One option is to employ auxiliary synthetic anomalies to improve the model training. However, synthetic anomalies may be of poor quality: anomalies that are unrealistic or indistinguishable from normal samples may deteriorate the detector's performance. Unfortunately, no existing methods quantify the quality of auxiliary anomalies. We fill in this gap and propose the expected anomaly posterior (EAP), an uncertainty-based score function that measures the quality of auxiliary anomalies by quantifying the total uncertainty of an anomaly detector. Experimentally on 40 benchmark datasets of images and tabular data, we show that EAP outperforms 12 adapted data quality estimators in the majority of cases.