Dual Representation Learning for Out-of-Distribution Detection

TL;DR

This paper introduces Dual Representation Learning (DRL), a method that enhances out-of-distribution detection by leveraging both label-related and distribution-related information from neural network representations.

Contribution

DRL is a novel approach that constructs complementary representations to improve the discrimination between in- and out-of-distribution samples.

Findings

DRL outperforms existing methods in OOD detection accuracy.

Combining label- and distribution-discriminative representations enhances detection.

Experiments validate the effectiveness of DRL on benchmark datasets.

Abstract

To classify in-distribution samples, deep neural networks explore strongly label-related information and discard weakly label-related information according to the information bottleneck. Out-of-distribution samples drawn from distributions differing from that of in-distribution samples could be assigned with unexpected high-confidence predictions because they could obtain minimum strongly label-related information. To distinguish in- and out-of-distribution samples, Dual Representation Learning (DRL) makes out-of-distribution samples harder to have high-confidence predictions by exploring both strongly and weakly label-related information from in-distribution samples. For a pretrained network exploring strongly label-related information to learn label-discriminative representations, DRL trains its auxiliary network exploring the remaining weakly label-related information to learn distribution-discriminative representations. Specifically, for a label-discriminative representation, DRL constructs its complementary distribution-discriminative representation by integrating diverse representations less similar to the label-discriminative representation. Accordingly, DRL combines label- and distribution-discriminative representations to detect out-of-distribution samples. Experiments show that DRL outperforms the state-of-the-art methods for out-of-distribution detection.