Mahalanobis++: Improving OOD Detection via Feature Normalization

TL;DR

This paper introduces Mahalanobis++, a method that normalizes features to improve out-of-distribution detection accuracy across various models by addressing Gaussian assumption violations.

Contribution

It proposes a simple $ ext{l}_2$-normalization technique to enhance Mahalanobis distance-based OOD detection, demonstrating consistent improvements across diverse models.

Findings

Normalization significantly improves OOD detection performance.

The method outperforms recent alternative approaches.

It is effective across multiple architectures and training schemes.

Abstract

Detecting out-of-distribution (OOD) examples is an important task for deploying reliable machine learning models in safety-critial applications. While post-hoc methods based on the Mahalanobis distance applied to pre-logit features are among the most effective for ImageNet-scale OOD detection, their performance varies significantly across models. We connect this inconsistency to strong variations in feature norms, indicating severe violations of the Gaussian assumption underlying the Mahalanobis distance estimation. We show that simple $\ell_2$-normalization of the features mitigates this problem effectively, aligning better with the premise of normally distributed data with shared covariance matrix. Extensive experiments on 44 models across diverse architectures and pretraining schemes show that $\ell_2$-normalization improves the conventional Mahalanobis distance-based approaches significantly and consistently, and outperforms other recently proposed OOD detection methods.