Can I trust my anomaly detection system? A case study based on explainable AI

TL;DR

This paper investigates the reliability of variational autoencoder-based anomaly detection systems by applying explainable AI techniques to reveal that many detections are based on misleading features, questioning their true effectiveness.

Contribution

It introduces a case study applying explainable AI to assess the robustness of VAE-based anomaly detection, highlighting potential pitfalls in current methods.

Findings

Many anomalies are detected due to misleading features.

Reconstruction disparities may obscure true anomaly causes.

Explainable AI provides new insights into detection reliability.

Abstract

Generative models based on variational autoencoders are a popular technique for detecting anomalies in images in a semi-supervised context. A common approach employs the anomaly score to detect the presence of anomalies, and it is known to reach high level of accuracy on benchmark datasets. However, since anomaly scores are computed from reconstruction disparities, they often obscure the detection of various spurious features, raising concerns regarding their actual efficacy. This case study explores the robustness of an anomaly detection system based on variational autoencoder generative models through the use of eXplainable AI methods. The goal is to get a different perspective on the real performances of anomaly detectors that use reconstruction differences. In our case study we discovered that, in many cases, samples are detected as anomalous for the wrong or misleading factors.