How to Overcome Curse-of-Dimensionality for Out-of-Distribution Detection?

TL;DR

This paper introduces Subspace Nearest Neighbor (SNN), a novel framework that improves out-of-distribution detection by addressing the curse-of-dimensionality through subspace regularization, leading to more effective distance measures.

Contribution

The paper proposes a new SNN framework that leverages relevant feature subspaces during training to enhance OOD detection in high-dimensional spaces.

Findings

SNN reduces FPR95 by 15.96% on CIFAR-100.

Subspace learning improves the distinguishability of ID and OOD data.

Comprehensive experiments validate the effectiveness of SNN.

Abstract

Machine learning models deployed in the wild can be challenged by out-of-distribution (OOD) data from unknown classes. Recent advances in OOD detection rely on distance measures to distinguish samples that are relatively far away from the in-distribution (ID) data. Despite the promise, distance-based methods can suffer from the curse-of-dimensionality problem, which limits the efficacy in high-dimensional feature space. To combat this problem, we propose a novel framework, Subspace Nearest Neighbor (SNN), for OOD detection. In training, our method regularizes the model and its feature representation by leveraging the most relevant subset of dimensions (i.e. subspace). Subspace learning yields highly distinguishable distance measures between ID and OOD data. We provide comprehensive experiments and ablations to validate the efficacy of SNN. Compared to the current best distance-based method, SNN reduces the average FPR95 by 15.96% on the CIFAR-100 benchmark.