Key Patches Are All You Need: A Multiple Instance Learning Framework For Robust Medical Diagnosis

TL;DR

This paper introduces a multiple instance learning framework that enhances the robustness and interpretability of medical diagnosis models by focusing on key image patches, reducing reliance on dataset biases.

Contribution

The proposed MIL framework limits information use to discriminative patches, improving robustness and interpretability in medical image diagnosis without sacrificing accuracy.

Findings

Maintains diagnostic accuracy on in-domain data

Improves robustness to demographic shifts

Provides detailed explanations of decision regions

Abstract

Deep learning models have revolutionized the field of medical image analysis, due to their outstanding performances. However, they are sensitive to spurious correlations, often taking advantage of dataset bias to improve results for in-domain data, but jeopardizing their generalization capabilities. In this paper, we propose to limit the amount of information these models use to reach the final classification, by using a multiple instance learning (MIL) framework. MIL forces the model to use only a (small) subset of patches in the image, identifying discriminative regions. This mimics the clinical procedures, where medical decisions are based on localized findings. We evaluate our framework on two medical applications: skin cancer diagnosis using dermoscopy and breast cancer diagnosis using mammography. Our results show that using only a subset of the patches does not compromise diagnostic performance for in-domain data, compared to the baseline approaches. However, our approach is more robust to shifts in patient demographics, while also providing more detailed explanations about which regions contributed to the decision. Code is available at: https://github.com/diogojpa99/MedicalMultiple-Instance-Learning.