Feature Bank Enhancement for Distance-based Out-of-Distribution Detection

TL;DR

This paper introduces Feature Bank Enhancement (FBE), a method that improves distance-based out-of-distribution detection by constraining extreme features, leading to state-of-the-art results on large-scale datasets.

Contribution

The paper proposes a novel FBE method that uses dataset statistics to mitigate feature bias in distance-based OOD detection, enhancing accuracy.

Findings

Achieves state-of-the-art OOD detection on ImageNet-1k and CIFAR-10.

Effectively constrains extreme features to improve separation boundaries.

Provides theoretical analysis supporting the method's effectiveness.

Abstract

Out-of-distribution (OOD) detection is critical to ensuring the reliability of deep learning applications and has attracted significant attention in recent years. A rich body of literature has emerged to develop efficient score functions that assign high scores to in-distribution (ID) samples and low scores to OOD samples, thereby helping distinguish OOD samples. Among these methods, distance-based score functions are widely used because of their efficiency and ease of use. However, deep learning often leads to a biased distribution of data features, and extreme features are inevitable. These extreme features make the distance-based methods tend to assign too low scores to ID samples. This limits the OOD detection capabilities of such methods. To address this issue, we propose a simple yet effective method, Feature Bank Enhancement (FBE), that uses statistical characteristics from dataset to identify and constrain extreme features to the separation boundaries, therapy making the distance between samples inside and outside the distribution farther. We conducted experiments on large-scale ImageNet-1k and CIFAR-10 respectively, and the results show that our method achieves state-of-the-art performance on both benchmark. Additionally, theoretical analysis and supplementary experiments are conducted to provide more insights into our method.