A Comparative Study of Machine Learning and Deep Learning for Out-of-Distribution Detection

TL;DR

This study compares machine learning and deep learning methods for out-of-distribution detection in medical imaging, finding ML approaches can match DL performance with lower computational costs in constrained settings.

Contribution

It provides a direct comparison showing ML can be as effective as DL for OOD detection in limited-variability medical images, emphasizing efficiency.

Findings

Both ML and DL achieved AUROC of 1.000 on datasets.

ML approach had lower latency than DL.

Lightweight ML approaches can match DL performance in constrained tasks.

Abstract

Out-of-distribution (OOD) detection is essential for building reliable AI systems, as models that produce outputs for invalid inputs cannot be trusted. Although deep learning (DL) is often assumed to outperform traditional machine learning (ML), medical imaging data are typically acquired under standardized protocols, leading to relatively constrained image variability in OOD detection tasks. This motivates a direct comparison between ML and DL approaches in this setting. The two approaches are evaluated on open datasets comprising over 60,000 fundus and non-fundus images across multiple resolutions. Both approaches achieved an AUROC of 1.000 and accuracies between 0.999 and 1.000 on internal and external validation sets, showing comparable detection performance. The ML approach, however, exhibited substantially lower end-to-end latency while maintaining equivalent accuracy, indicating greater computational efficiency. These results suggest that for OOD detection tasks of limited visual complexity, lightweight ML approaches can achieve DL-level performance with significantly reduced computational cost, supporting practical real-world deployment.