Three Factors to Improve Out-of-Distribution Detection

TL;DR

This paper proposes three strategies—self-knowledge distillation, semi-hard outlier sampling, and supervised contrastive learning—to simultaneously improve out-of-distribution detection and classification accuracy, addressing the traditional trade-off.

Contribution

The paper introduces a novel combination of techniques that enhance both OOD detection and classification accuracy without compromising either.

Findings

Improved OOD detection metrics (AUROC, FPR, AUPR)

Enhanced classification accuracy (ACC)

Achieved state-of-the-art performance on benchmarks

Abstract

In the problem of out-of-distribution (OOD) detection, the usage of auxiliary data as outlier data for fine-tuning has demonstrated encouraging performance. However, previous methods have suffered from a trade-off between classification accuracy (ACC) and OOD detection performance (AUROC, FPR, AUPR). To improve this trade-off, we make three contributions: (i) Incorporating a self-knowledge distillation loss can enhance the accuracy of the network; (ii) Sampling semi-hard outlier data for training can improve OOD detection performance with minimal impact on accuracy; (iii) The introduction of our novel supervised contrastive learning can simultaneously improve OOD detection performance and the accuracy of the network. By incorporating all three factors, our approach enhances both accuracy and OOD detection performance by addressing the trade-off between classification and OOD detection. Our method achieves improvements over previous approaches in both performance metrics.