Understanding the Feature Norm for Out-of-Distribution Detection

TL;DR

This paper investigates why neural network feature norms differ between in-distribution and out-of-distribution samples, revealing their relation to classifier confidence and proposing a new norm to improve OOD detection.

Contribution

The study uncovers the link between feature norm and classifier confidence, introduces a class-agnostic norm, and proposes a novel negative-aware norm for better OOD detection.

Findings

Feature norm correlates with classifier confidence (max logit).

Feature norm can detect OOD samples across models.

Negative-aware norm improves OOD detection performance.

Abstract

A neural network trained on a classification dataset often exhibits a higher vector norm of hidden layer features for in-distribution (ID) samples, while producing relatively lower norm values on unseen instances from out-of-distribution (OOD). Despite this intriguing phenomenon being utilized in many applications, the underlying cause has not been thoroughly investigated. In this study, we demystify this very phenomenon by scrutinizing the discriminative structures concealed in the intermediate layers of a neural network. Our analysis leads to the following discoveries: (1) The feature norm is a confidence value of a classifier hidden in the network layer, specifically its maximum logit. Hence, the feature norm distinguishes OOD from ID in the same manner that a classifier confidence does. (2) The feature norm is class-agnostic, thus it can detect OOD samples across diverse discriminative models. (3) The conventional feature norm fails to capture the deactivation tendency of hidden layer neurons, which may lead to misidentification of ID samples as OOD instances. To resolve this drawback, we propose a novel negative-aware norm (NAN) that can capture both the activation and deactivation tendencies of hidden layer neurons. We conduct extensive experiments on NAN, demonstrating its efficacy and compatibility with existing OOD detectors, as well as its capability in label-free environments.