Model2Detector: Widening the Information Bottleneck for Out-of-Distribution Detection using a Handful of Gradient Steps

TL;DR

This paper introduces Model2Detector, a post-processing method that transforms trained neural networks into effective out-of-distribution detectors using minimal gradient steps, improving detection accuracy and reducing computational costs.

Contribution

It proposes a novel information-theoretic approach and a simple gradient-based post-processing technique to enhance OOD detection without additional training.

Findings

Outperforms state-of-the-art OOD detection methods on image datasets

Reduces computational complexity compared to existing approaches

Enables conversion of trained models into OOD detectors with minimal steps

Abstract

Out-of-distribution detection is an important capability that has long eluded vanilla neural networks. Deep Neural networks (DNNs) tend to generate over-confident predictions when presented with inputs that are significantly out-of-distribution (OOD). This can be dangerous when employing machine learning systems in the wild as detecting attacks can thus be difficult. Recent advances inference-time out-of-distribution detection help mitigate some of these problems. However, existing methods can be restrictive as they are often computationally expensive. Additionally, these methods require training of a downstream detector model which learns to detect OOD inputs from in-distribution ones. This, therefore, adds latency during inference. Here, we offer an information theoretic perspective on why neural networks are inherently incapable of OOD detection. We attempt to mitigate these flaws by converting a trained model into a an OOD detector using a handful of steps of gradient descent. Our work can be employed as a post-processing method whereby an inference-time ML system can convert a trained model into an OOD detector. Experimentally, we show how our method consistently outperforms the state-of-the-art in detection accuracy on popular image datasets while also reducing computational complexity.