Iterative Deployment Exposure for Unsupervised Out-of-Distribution Detection

TL;DR

This paper introduces a realistic setting for unsupervised out-of-distribution detection during deployment, proposing a method that iteratively refines detection using observed data, significantly improving performance in medical imaging tasks.

Contribution

The paper presents IDE, a new deployment scenario for U-OOD detection, and introduces CSO, a method that refines OOD detection iteratively using unlabeled data and a novel scoring function.

Findings

Significant performance improvements over baselines on medical imaging datasets.

Effective detection of OOD samples in a realistic deployment scenario.

Robustness of the method with limited OOD examples.

Abstract

Deep learning models are vulnerable to performance degradation when encountering out-of-distribution (OOD) images, potentially leading to misdiagnoses and compromised patient care. These shortcomings have led to great interest in the field of OOD detection. Existing unsupervised OOD (U-OOD) detection methods typically assume that OOD samples originate from an unconcentrated distribution complementary to the training distribution, neglecting the reality that deployed models passively accumulate task-specific OOD samples over time. To better reflect this real-world scenario, we introduce Iterative Deployment Exposure (IDE), a novel and more realistic setting for U-OOD detection. We propose CSO, a method for IDE that starts from a U-OOD detector that is agnostic to the OOD distribution and slowly refines it during deployment using observed unlabeled data. CSO uses a new U-OOD scoring function that combines the Mahalanobis distance with a nearest-neighbor approach, along with a novel confidence-scaled few-shot OOD detector to effectively learn from limited OOD examples. We validate our approach on a dedicated benchmark, showing that our method greatly improves upon strong baselines on three medical imaging modalities.