Provable Guarantees for Understanding Out-of-distribution Detection

TL;DR

This paper develops a theoretical framework for out-of-distribution detection, introduces a new method GEM with proven guarantees, and demonstrates its superior empirical performance on CIFAR-100.

Contribution

It provides the first unified theoretical understanding of OOD detection and proposes GEM, a novel method with provable guarantees and improved empirical results.

Findings

GEM outperforms baseline by 16.57% FPR95 on CIFAR-100.

Provides formal guarantees linking data distribution properties to OOD detection performance.

Unifies theoretical understanding of OOD detection methods.

Abstract

Out-of-distribution (OOD) detection is important for deploying machine learning models in the real world, where test data from shifted distributions can naturally arise. While a plethora of algorithmic approaches have recently emerged for OOD detection, a critical gap remains in theoretical understanding. In this work, we develop an analytical framework that characterizes and unifies the theoretical understanding for OOD detection. Our analytical framework motivates a novel OOD detection method for neural networks, GEM, which demonstrates both theoretical and empirical superiority. In particular, on CIFAR-100 as in-distribution data, our method outperforms a competitive baseline by 16.57% (FPR95). Lastly, we formally provide provable guarantees and comprehensive analysis of our method, underpinning how various properties of data distribution affect the performance of OOD detection.