Selecting the Best Sequential Transfer Path for Medical Image Segmentation with Limited Labeled Data

TL;DR

This paper introduces a novel sequential transfer learning method with a task affinity metric to optimize source task selection for medical image segmentation, significantly improving performance with limited labeled data.

Contribution

It proposes a task affinity-based sequential transfer strategy that identifies the optimal transfer path among multiple source tasks for medical image segmentation.

Findings

Achieved an average of 2.58% Dice score improvement across datasets.

Significant 6.00% Dice score gain on FeTS 2022 dataset.

Validated effectiveness on three MRI datasets.

Abstract

The medical image processing field often encounters the critical issue of scarce annotated data. Transfer learning has emerged as a solution, yet how to select an adequate source task and effectively transfer the knowledge to the target task remains challenging. To address this, we propose a novel sequential transfer scheme with a task affinity metric tailored for medical images. Considering the characteristics of medical image segmentation tasks, we analyze the image and label similarity between tasks and compute the task affinity scores, which assess the relatedness among tasks. Based on this, we select appropriate source tasks and develop an effective sequential transfer strategy by incorporating intermediate source tasks to gradually narrow the domain discrepancy and minimize the transfer cost. Thereby we identify the best sequential transfer path for the given target task. Extensive experiments on three MRI medical datasets, FeTS 2022, iSeg-2019, and WMH, demonstrate the efficacy of our method in finding the best source sequence. Compared with directly transferring from a single source task, the sequential transfer results underline a significant improvement in target task performance, achieving an average of 2.58% gain in terms of segmentation Dice score, notably, 6.00% for FeTS 2022. Code is available at the git repository.