Transfer learning from a sparsely annotated dataset of 3D medical images

TL;DR

This study demonstrates that transfer learning from a large, sparsely annotated 3D medical imaging dataset significantly improves organ segmentation performance in new tasks, especially with limited data, and is beneficial across different imaging modalities.

Contribution

It introduces a transfer learning approach using a base 3D U-Net model trained on sparse data, showing its effectiveness in improving segmentation accuracy in medical imaging tasks.

Findings

Transfer learning boosts performance with small datasets.

Fine-tuning the base model yields better results than full retraining.

Cross-modality transfer learning from CT scans is effective.

Abstract

Transfer learning leverages pre-trained model features from a large dataset to save time and resources when training new models for various tasks, potentially enhancing performance. Due to the lack of large datasets in the medical imaging domain, transfer learning from one medical imaging model to other medical imaging models has not been widely explored. This study explores the use of transfer learning to improve the performance of deep convolutional neural networks for organ segmentation in medical imaging. A base segmentation model (3D U-Net) was trained on a large and sparsely annotated dataset; its weights were used for transfer learning on four new down-stream segmentation tasks for which a fully annotated dataset was available. We analyzed the training set size's influence to simulate scarce data. The results showed that transfer learning from the base model was beneficial when small datasets were available, providing significant performance improvements; where fine-tuning the base model is more beneficial than updating all the network weights with vanilla transfer learning. Transfer learning with fine-tuning increased the performance by up to 0.129 (+28\%) Dice score than experiments trained from scratch, and on average 23 experiments increased the performance by 0.029 Dice score in the new segmentation tasks. The study also showed that cross-modality transfer learning using CT scans was beneficial. The findings of this study demonstrate the potential of transfer learning to improve the efficiency of annotation and increase the accessibility of accurate organ segmentation in medical imaging, ultimately leading to improved patient care. We made the network definition and weights publicly available to benefit other users and researchers.