Unmasking unlearnable models: a classification challenge for biomedical images without visible cues

TL;DR

This paper investigates why current models struggle to predict biomedical traits from images lacking visual cues, revealing their unlearnability and the need for new architectures to improve real-world applications.

Contribution

The study benchmarks existing models, analyzes their limitations, and proposes feature selection to enhance interpretability, highlighting the necessity for novel architectures.

Findings

Current models are unlearnable for non-visible cues

Benchmarking reveals suboptimal performance of existing models

Feature selection improves interpretability

Abstract

Predicting traits from images lacking visual cues is challenging, as algorithms are designed to capture visually correlated ground truth. This problem is critical in biomedical sciences, and their solution can improve the efficacy of non-invasive methods. For example, a recent challenge of predicting MGMT methylation status from MRI images is critical for treatment decisions of glioma patients. Using less robust models poses a significant risk in these critical scenarios and underscores the urgency of addressing this issue. Despite numerous efforts, contemporary models exhibit suboptimal performance, and underlying reasons for this limitation remain elusive. In this study, we demystify the complexity of MGMT status prediction through a comprehensive exploration by performing benchmarks of existing models adjoining transfer learning. Their architectures were further dissected by observing gradient flow across layers. Additionally, a feature selection strategy was applied to improve model interpretability. Our finding highlighted that current models are unlearnable and may require new architectures to explore applications in the real world. We believe our study will draw immediate attention and catalyse advancements in predictive modelling with non-visible cues.