Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks

TL;DR

This paper investigates how soft labeling influences out-of-distribution detection in deep neural networks, revealing that it can either improve or deteriorate OOD detection depending on regularization strategies, and proposing a future direction for OOD-robust models.

Contribution

It demonstrates the impact of soft labeling on OOD detection performance and suggests that proper regularization can enhance robustness without extra training or model modifications.

Findings

Soft labeling can both improve and worsen OOD detection.

Proper regularization with soft labels can enhance OOD robustness.

Potential for improving classification accuracy without additional OOD training.

Abstract

Soft labeling becomes a common output regularization for generalization and model compression of deep neural networks. However, the effect of soft labeling on out-of-distribution (OOD) detection, which is an important topic of machine learning safety, is not explored. In this study, we show that soft labeling can determine OOD detection performance. Specifically, how to regularize outputs of incorrect classes by soft labeling can deteriorate or improve OOD detection. Based on the empirical results, we postulate a future work for OOD-robust DNNs: a proper output regularization by soft labeling can construct OOD-robust DNNs without additional training of OOD samples or modifying the models, while improving classification accuracy.