Deep Residual Flow for Out of Distribution Detection

TL;DR

This paper introduces a novel residual flow architecture that models feature distributions more expressively for out-of-distribution detection, significantly improving detection accuracy over existing Gaussian-based methods.

Contribution

The paper proposes the residual flow, a new flow-based model that enhances out-of-distribution detection by learning residual distributions, outperforming current state-of-the-art Gaussian models.

Findings

Improves TNR from 56.7% to 77.5% at 95% TPR on CIFAR-100 vs. ImageNet.

Effective across ResNet and DenseNet architectures.

Provides a principled, general approach for Gaussian-approximated data.

Abstract

The effective application of neural networks in the real-world relies on proficiently detecting out-of-distribution examples. Contemporary methods seek to model the distribution of feature activations in the training data for adequately distinguishing abnormalities, and the state-of-the-art method uses Gaussian distribution models. In this work, we present a novel approach that improves upon the state-of-the-art by leveraging an expressive density model based on normalizing flows. We introduce the residual flow, a novel flow architecture that learns the residual distribution from a base Gaussian distribution. Our model is general, and can be applied to any data that is approximately Gaussian. For out of distribution detection in image datasets, our approach provides a principled improvement over the state-of-the-art. Specifically, we demonstrate the effectiveness of our method in ResNet and DenseNet architectures trained on various image datasets. For example, on a ResNet trained on CIFAR-100 and evaluated on detection of out-of-distribution samples from the ImageNet dataset, holding the true positive rate (TPR) at $95\%$, we improve the true negative rate (TNR) from $56.7\%$ (current state-of-the-art) to $77.5\%$ (ours).