Redesigning Out-of-Distribution Detection on 3D Medical Images

TL;DR

This paper introduces a new evaluation metric, Expected Performance Drop (EPD), for OOD detection in 3D medical images, focusing on clinical impact rather than artificial distinctions, and demonstrates its effectiveness across multiple CT and MRI challenges.

Contribution

The paper redefines OOD detection for medical imaging using downstream model performance as a pseudometric and introduces EPD as a novel evaluation metric.

Findings

EPD effectively ranks OOD detection methods by clinical impact.

EPD-based evaluation outperforms traditional metrics in 11 CT and MRI challenges.

Redefining OOD detection improves clinical relevance of model assessments.

Abstract

Detecting out-of-distribution (OOD) samples for trusted medical image segmentation remains a significant challenge. The critical issue here is the lack of a strict definition of abnormal data, which often results in artificial problem settings without measurable clinical impact. In this paper, we redesign the OOD detection problem according to the specifics of volumetric medical imaging and related downstream tasks (e.g., segmentation). We propose using the downstream model's performance as a pseudometric between images to define abnormal samples. This approach enables us to weigh different samples based on their performance impact without an explicit ID/OOD distinction. We incorporate this weighting in a new metric called Expected Performance Drop (EPD). EPD is our core contribution to the new problem design, allowing us to rank methods based on their clinical impact. We demonstrate the effectiveness of EPD-based evaluation in 11 CT and MRI OOD detection challenges.