Towards In-distribution Compatibility in Out-of-distribution Detection

TL;DR

This paper introduces a novel out-of-distribution detection method that enhances in-distribution compatibility, leading to improved detection accuracy and in-distribution classification performance by addressing key incompatibility issues.

Contribution

It identifies causes of in-distribution incompatibility and proposes a new method that adapts model design and loss functions to mitigate interference with in-distribution features.

Findings

Achieves state-of-the-art OOD detection performance

Improves in-distribution accuracy

Reduces interference with in-distribution features

Abstract

Deep neural network, despite its remarkable capability of discriminating targeted in-distribution samples, shows poor performance on detecting anomalous out-of-distribution data. To address this defect, state-of-the-art solutions choose to train deep networks on an auxiliary dataset of outliers. Various training criteria for these auxiliary outliers are proposed based on heuristic intuitions. However, we find that these intuitively designed outlier training criteria can hurt in-distribution learning and eventually lead to inferior performance. To this end, we identify three causes of the in-distribution incompatibility: contradictory gradient, false likelihood, and distribution shift. Based on our new understandings, we propose a new out-of-distribution detection method by adapting both the top-design of deep models and the loss function. Our method achieves in-distribution compatibility by pursuing less interference with the probabilistic characteristic of in-distribution features. On several benchmarks, our method not only achieves the state-of-the-art out-of-distribution detection performance but also improves the in-distribution accuracy.