On dataset transferability in medical image classification

TL;DR

This paper introduces a new transferability metric tailored for medical image classification, addressing limitations of existing methods by combining feature quality and gradients, and demonstrates its superior performance in various transfer scenarios.

Contribution

The authors propose a novel transferability metric that improves evaluation accuracy for medical image tasks and provide comprehensive benchmarking and insights into transfer dynamics.

Findings

Our metric outperforms existing methods in transferability estimation.

It effectively evaluates source dataset suitability for medical image classification.

Insights into cross-domain transfer from natural to medical images are provided.

Abstract

Current transferability estimation methods designed for natural image datasets are often suboptimal in medical image classification. These methods primarily focus on estimating the suitability of pre-trained source model features for a target dataset, which can lead to unrealistic predictions, such as suggesting that the target dataset is the best source for itself. To address this, we propose a novel transferability metric that combines feature quality with gradients to evaluate both the suitability and adaptability of source model features for target tasks. We evaluate our approach in two new scenarios: source dataset transferability for medical image classification and cross-domain transferability. Our results show that our method outperforms existing transferability metrics in both settings. We also provide insight into the factors influencing transfer performance in medical image classification, as well as the dynamics of cross-domain transfer from natural to medical images. Additionally, we provide ground-truth transfer performance benchmarking results to encourage further research into transferability estimation for medical image classification. Our code and experiments are available at https://github.com/DovileDo/transferability-in-medical-imaging.