When Accuracy Is Not Enough: Uncertainty Collapse between Noisy Label Learning and Out-of-Distribution Detection

TL;DR

This paper reveals that high accuracy in noisy label learning does not guarantee reliable out-of-distribution detection due to uncertainty collapse, and proposes Virtual Margin Regularization to mitigate this issue.

Contribution

It introduces an ACC-OOD benchmark for evaluating noisy label learners on OOD detection and proposes VMR to improve OOD reliability without sacrificing accuracy.

Findings

High closed-set accuracy does not ensure OOD reliability.

Uncertainty collapse causes overlap between in-distribution errors and OOD inputs.

VMR partially mitigates the collapse and improves OOD detection.

Abstract

Learning with noisy labels (LNL) is typically benchmarked by closed-set classification accuracy, yet deployment often requires classifiers to reject out-of-distribution (OOD) inputs. We present a learner-agnostic ACC-OOD benchmark that freezes LNL checkpoints and evaluates them with standardized near-/far-OOD routing and post-hoc scores across synthetic and real label noise. The benchmark reveals a recurring failure mode: high closed-set accuracy does not ensure OOD reliability, because low-confidence, misclassified in-distribution samples can overlap the score and feature regions occupied by OOD inputs under noisy training. We term this pathology uncertainty collapse. This structural overlap can make high-accuracy LNL methods lose separability at the ID-error/OOD interface under standard OOD scores. As an intervention, we study Virtual Margin Regularization (VMR), a lightweight repair probe demonstrated mainly with PSSCL that synthesizes boundary virtual outliers on trusted ID batches and widens the energy margin. VMR partially reduces the collapse-induced far-OOD failure without replacing the host objective or sacrificing closed-set accuracy in the tested settings. These results support LNL benchmarks that co-report closed-set generalization, open-world reliability, and structural overlap diagnostics.