Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation

TL;DR

This paper introduces a novel transferability estimation method tailored for medical image segmentation, enabling better selection of pre-trained models to improve transfer learning efficiency.

Contribution

We propose a source-free transferability estimation framework that addresses limitations of existing methods, considering class consistency and feature variety for medical image segmentation.

Findings

Our method outperforms existing transferability estimation algorithms.

Extensive experiments validate the effectiveness of the proposed approach.

The framework enhances model selection for medical image segmentation tasks.

Abstract

Transfer learning is a critical technique in training deep neural networks for the challenging medical image segmentation task that requires enormous resources. With the abundance of medical image data, many research institutions release models trained on various datasets that can form a huge pool of candidate source models to choose from. Hence, it's vital to estimate the source models' transferability (i.e., the ability to generalize across different downstream tasks) for proper and efficient model reuse. To make up for its deficiency when applying transfer learning to medical image segmentation, in this paper, we therefore propose a new Transferability Estimation (TE) method. We first analyze the drawbacks of using the existing TE algorithms for medical image segmentation and then design a source-free TE framework that considers both class consistency and feature variety for better estimation. Extensive experiments show that our method surpasses all current algorithms for transferability estimation in medical image segmentation. Code is available at https://github.com/EndoluminalSurgicalVision-IMR/CCFV