Multi-layer Aggregation as a key to feature-based OOD detection

TL;DR

This paper demonstrates that multi-layer feature aggregation significantly improves out-of-distribution detection in medical imaging, outperforming single-layer methods and highlighting the importance of network architecture.

Contribution

It provides a comprehensive comparison of feature-based OOD detection methods, emphasizing the effectiveness of multi-layer aggregation across diverse anomalies and architectures.

Findings

Multi-layer methods outperform single-layer approaches.

OOD detection performance varies with neural network architecture.

Multi-layer aggregation provides more consistent anomaly detection.

Abstract

Deep Learning models are easily disturbed by variations in the input images that were not observed during the training stage, resulting in unpredictable predictions. Detecting such Out-of-Distribution (OOD) images is particularly crucial in the context of medical image analysis, where the range of possible abnormalities is extremely wide. Recently, a new category of methods has emerged, based on the analysis of the intermediate features of a trained model. These methods can be divided into 2 groups: single-layer methods that consider the feature map obtained at a fixed, carefully chosen layer, and multi-layer methods that consider the ensemble of the feature maps generated by the model. While promising, a proper comparison of these algorithms is still lacking. In this work, we compared various feature-based OOD detection methods on a large spectra of OOD (20 types), representing approximately 7800 3D MRIs. Our experiments shed the light on two phenomenons. First, multi-layer methods consistently outperform single-layer approaches, which tend to have inconsistent behaviour depending on the type of anomaly. Second, the OOD detection performance highly depends on the architecture of the underlying neural network.