Your Classifier Can Be Secretly a Likelihood-Based OOD Detector

TL;DR

This paper introduces INK, a likelihood-based score for discriminative classifiers that improves out-of-distribution detection by providing a rigorous likelihood interpretation, achieving state-of-the-art results on multiple benchmarks.

Contribution

The paper proposes INK, a novel likelihood interpretation for discriminative classifiers using hyperspherical embeddings, bridging the gap between discriminative and generative approaches for OOD detection.

Findings

INK achieves state-of-the-art OOD detection performance.

It performs well on both far-OOD and near-OOD scenarios.

The method is validated on the OpenOOD benchmark.

Abstract

The ability to detect out-of-distribution (OOD) inputs is critical to guarantee the reliability of classification models deployed in an open environment. A fundamental challenge in OOD detection is that a discriminative classifier is typically trained to estimate the posterior probability p(y|z) for class y given an input z, but lacks the explicit likelihood estimation of p(z) ideally needed for OOD detection. While numerous OOD scoring functions have been proposed for classification models, these estimate scores are often heuristic-driven and cannot be rigorously interpreted as likelihood. To bridge the gap, we propose Intrinsic Likelihood (INK), which offers rigorous likelihood interpretation to modern discriminative-based classifiers. Specifically, our proposed INK score operates on the constrained latent embeddings of a discriminative classifier, which are modeled as a mixture of hyperspherical embeddings with constant norm. We draw a novel connection between the hyperspherical distribution and the intrinsic likelihood, which can be effectively optimized in modern neural networks. Extensive experiments on the OpenOOD benchmark empirically demonstrate that INK establishes a new state-of-the-art in a variety of OOD detection setups, including both far-OOD and near-OOD. Code is available at https://github.com/deeplearning-wisc/ink.