DOODLER: Determining Out-Of-Distribution Likelihood from Encoder Reconstructions

TL;DR

DOODLER is a novel OOD detection method using VAEs trained on the same data as the main model, effectively identifying out-of-distribution inputs with pixel-wise localization, requiring minimal assumptions about OOD data.

Contribution

It introduces a VAE-based approach for OOD detection that requires only weak assumptions about OOD data and provides pixel-wise OOD likelihood segmentations.

Findings

Matches or outperforms existing methods in OOD detection accuracy.

Enables pixel-wise OOD likelihood segmentation.

Requires minimal assumptions about OOD datasets.

Abstract

Deep Learning models possess two key traits that, in combination, make their use in the real world a risky prospect. One, they do not typically generalize well outside of the distribution for which they were trained, and two, they tend to exhibit confident behavior regardless of whether or not they are producing meaningful outputs. While Deep Learning possesses immense power to solve realistic, high-dimensional problems, these traits in concert make it difficult to have confidence in their real-world applications. To overcome this difficulty, the task of Out-Of-Distribution (OOD) Detection has been defined, to determine when a model has received an input from outside of the distribution for which it is trained to operate. This paper introduces and examines a novel methodology, DOODLER, for OOD Detection, which directly leverages the traits which result in its necessity. By training a Variational Auto-Encoder (VAE) on the same data as another Deep Learning model, the VAE learns to accurately reconstruct In-Distribution (ID) inputs, but not to reconstruct OOD inputs, meaning that its failure state can be used to perform OOD Detection. Unlike other work in the area, DOODLER requires only very weak assumptions about the existence of an OOD dataset, allowing for more realistic application. DOODLER also enables pixel-wise segmentations of input images by OOD likelihood, and experimental results show that it matches or outperforms methodologies that operate under the same constraints.