Know Your Space: Inlier and Outlier Construction for Calibrating Medical OOD Detectors

TL;DR

This paper investigates how the choice of data synthesis space impacts the calibration of medical OOD detectors, demonstrating that latent-space inliers combined with pixel-space outliers significantly improve detection performance.

Contribution

It reveals the importance of synthesis space in calibrating OOD detectors and proposes an effective protocol using latent-space inliers and pixel-space outliers for medical imaging.

Findings

Achieves 15-35% improvement in AUROC over state-of-the-art methods.

Synthesizing inliers in latent space and outliers in pixel space enhances calibration.

Empirical validation across multiple medical imaging benchmarks.

Abstract

We focus on the problem of producing well-calibrated out-of-distribution (OOD) detectors, in order to enable safe deployment of medical image classifiers. Motivated by the difficulty of curating suitable calibration datasets, synthetic augmentations have become highly prevalent for inlier/outlier specification. While there have been rapid advances in data augmentation techniques, this paper makes a striking finding that the space in which the inliers and outliers are synthesized, in addition to the type of augmentation, plays a critical role in calibrating OOD detectors. Using the popular energy-based OOD detection framework, we find that the optimal protocol is to synthesize latent-space inliers along with diverse pixel-space outliers. Based on empirical studies with multiple medical imaging benchmarks, we demonstrate that our approach consistently leads to superior OOD detection ($15\% - 35\%$ in AUROC) over the state-of-the-art in a variety of open-set recognition settings.