A Closer Look at the Learnability of Out-of-Distribution (OOD) Detection

TL;DR

This paper investigates the theoretical learnability of out-of-distribution detection in machine learning, distinguishing between uniform and non-uniform learnability, and provides algorithms and complexity analysis where learnable.

Contribution

It introduces a PAC learning framework for OOD detection, clarifies conditions for learnability, and offers algorithms with sample complexity analysis.

Findings

Non-uniform learnability can turn negative results into positive.

Conditions for uniform and non-uniform learnability are characterized.

Concrete algorithms with sample complexity bounds are provided for learnable cases.

Abstract

Machine learning algorithms often encounter different or "out-of-distribution" (OOD) data at deployment time, and OOD detection is frequently employed to detect these examples. While it works reasonably well in practice, existing theoretical results on OOD detection are highly pessimistic. In this work, we take a closer look at this problem, and make a distinction between uniform and non-uniform learnability, following PAC learning theory. We characterize under what conditions OOD detection is uniformly and non-uniformly learnable, and we show that in several cases, non-uniform learnability turns a number of negative results into positive. In all cases where OOD detection is learnable, we provide concrete learning algorithms and a sample-complexity analysis.