Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data

TL;DR

This paper introduces Generalized ODIN, a method for out-of-distribution detection in images that does not require OoD data for tuning, improving performance and analyzing different types of distribution shifts.

Contribution

It extends ODIN by proposing two tuning-free strategies for OoD detection, enhancing robustness without relying on OoD data during training or tuning.

Findings

Decomposition of confidence scoring improves OoD detection.

Modified input pre-processing enhances detection performance.

Semantic and non-semantic shifts differ in detection difficulty.

Abstract

Deep neural networks have attained remarkable performance when applied to data that comes from the same distribution as that of the training set, but can significantly degrade otherwise. Therefore, detecting whether an example is out-of-distribution (OoD) is crucial to enable a system that can reject such samples or alert users. Recent works have made significant progress on OoD benchmarks consisting of small image datasets. However, many recent methods based on neural networks rely on training or tuning with both in-distribution and out-of-distribution data. The latter is generally hard to define a-priori, and its selection can easily bias the learning. We base our work on a popular method ODIN, proposing two strategies for freeing it from the needs of tuning with OoD data, while improving its OoD detection performance. We specifically propose to decompose confidence scoring as well as a modified input pre-processing method. We show that both of these significantly help in detection performance. Our further analysis on a larger scale image dataset shows that the two types of distribution shifts, specifically semantic shift and non-semantic shift, present a significant difference in the difficulty of the problem, providing an analysis of when ODIN-like strategies do or do not work.