Why is the Mahalanobis Distance Effective for Anomaly Detection?

TL;DR

This paper investigates why Mahalanobis distance-based confidence scores excel in anomaly detection, revealing that their success is due to information unrelated to classification confidence, and proposes combining it with ODIN for improved detection.

Contribution

The study analyzes the reasons behind Mahalanobis distance's effectiveness, challenges its assumed theoretical basis, and introduces a combined method with ODIN for enhanced performance.

Findings

Mahalanobis score's success is due to non-classification information.

Its performance surpasses expectations based on theoretical assumptions.

Combining Mahalanobis with ODIN improves anomaly detection robustness.

Abstract

The Mahalanobis distance-based confidence score, a recently proposed anomaly detection method for pre-trained neural classifiers, achieves state-of-the-art performance on both out-of-distribution (OoD) and adversarial examples detection. This work analyzes why this method exhibits such strong performance in practical settings while imposing an implausible assumption; namely, that class conditional distributions of pre-trained features have tied covariance. Although the Mahalanobis distance-based method is claimed to be motivated by classification prediction confidence, we find that its superior performance stems from information not useful for classification. This suggests that the reason the Mahalanobis confidence score works so well is mistaken, and makes use of different information from ODIN, another popular OoD detection method based on prediction confidence. This perspective motivates us to combine these two methods, and the combined detector exhibits improved performance and robustness. These findings provide insight into the behavior of neural classifiers in response to anomalous inputs.