Finding the Most Transferable Tasks for Brain Image Segmentation

TL;DR

This paper introduces a framework for selecting optimal source tasks for transfer learning in brain image segmentation, leveraging modality, region of interest, and transferability metrics to enhance performance.

Contribution

It proposes a novel prior knowledge guided, transferability-based method for source task selection, improving transfer learning effectiveness in medical image segmentation.

Findings

Transfer from the same modality generally yields better results.

Stronger RoI shape similarity improves transfer performance.

Filtering source tasks based on modality and RoI enhances transferability estimation.

Abstract

Although many studies have successfully applied transfer learning to medical image segmentation, very few of them have investigated the selection strategy when multiple source tasks are available for transfer. In this paper, we propose a prior knowledge guided and transferability based framework to select the best source tasks among a collection of brain image segmentation tasks, to improve the transfer learning performance on the given target task. The framework consists of modality analysis, RoI (region of interest) analysis, and transferability estimation, such that the source task selection can be refined step by step. Specifically, we adapt the state-of-the-art analytical transferability estimation metrics to medical image segmentation tasks and further show that their performance can be significantly boosted by filtering candidate source tasks based on modality and RoI characteristics. Our experiments on brain matter, brain tumor, and white matter hyperintensities segmentation datasets reveal that transferring from different tasks under the same modality is often more successful than transferring from the same task under different modalities. Furthermore, within the same modality, transferring from the source task that has stronger RoI shape similarity with the target task can significantly improve the final transfer performance. And such similarity can be captured using the Structural Similarity index in the label space.