Representation Norm Amplification for Out-of-Distribution Detection in Long-Tail Learning

TL;DR

This paper introduces Representation Norm Amplification (RNA), a novel method that improves out-of-distribution detection in long-tailed datasets by leveraging representation norms to better distinguish OOD samples from in-distribution data.

Contribution

RNA decouples OOD detection from in-distribution classification by amplifying the representation norm discrepancy, leading to superior performance over existing methods.

Findings

RNA outperforms state-of-the-art in OOD detection and classification accuracy.

RNA achieves 1.70% and 9.46% improvements in FPR95 on CIFAR10-LT and ImageNet-LT.

RNA improves classification accuracy by 2.43% and 6.87% on CIFAR10-LT and ImageNet-LT.

Abstract

Detecting out-of-distribution (OOD) samples is a critical task for reliable machine learning. However, it becomes particularly challenging when the models are trained on long-tailed datasets, as the models often struggle to distinguish tail-class in-distribution samples from OOD samples. We examine the main challenges in this problem by identifying the trade-offs between OOD detection and in-distribution (ID) classification, faced by existing methods. We then introduce our method, called \textit{Representation Norm Amplification} (RNA), which solves this challenge by decoupling the two problems. The main idea is to use the norm of the representation as a new dimension for OOD detection, and to develop a training method that generates a noticeable discrepancy in the representation norm between ID and OOD data, while not perturbing the feature learning for ID classification. Our experiments show that RNA achieves superior performance in both OOD detection and classification compared to the state-of-the-art methods, by 1.70\% and 9.46\% in FPR95 and 2.43\% and 6.87\% in classification accuracy on CIFAR10-LT and ImageNet-LT, respectively. The code for this work is available at https://github.com/dgshin21/RNA.