Out of Distribution Detection via Neural Network Anchoring

TL;DR

This paper introduces a novel heteroscedastic temperature scaling method with an anchoring training strategy for out-of-distribution detection, achieving state-of-the-art results without needing outlier datasets or ensembling.

Contribution

It proposes a new training strategy called anchoring that estimates sample-specific temperature values, improving OOD detection performance and linking temperature estimates to epistemic uncertainty.

Findings

Achieves state-of-the-art OOD detection across benchmarks.

Does not require outlier datasets or ensembling.

Effective for far, near, and semantically coherent OOD detection.

Abstract

Our goal in this paper is to exploit heteroscedastic temperature scaling as a calibration strategy for out of distribution (OOD) detection. Heteroscedasticity here refers to the fact that the optimal temperature parameter for each sample can be different, as opposed to conventional approaches that use the same value for the entire distribution. To enable this, we propose a new training strategy called anchoring that can estimate appropriate temperature values for each sample, leading to state-of-the-art OOD detection performance across several benchmarks. Using NTK theory, we show that this temperature function estimate is closely linked to the epistemic uncertainty of the classifier, which explains its behavior. In contrast to some of the best-performing OOD detection approaches, our method does not require exposure to additional outlier datasets, custom calibration objectives, or model ensembling. Through empirical studies with different OOD detection settings -- far OOD, near OOD, and semantically coherent OOD - we establish a highly effective OOD detection approach. Code to reproduce our results is available at github.com/LLNL/AMP