Med3D: Transfer Learning for 3D Medical Image Analysis

TL;DR

This paper introduces Med3D, a transfer learning approach using a large, diverse 3D medical dataset to pre-train models, significantly improving training speed and accuracy in various 3D medical imaging tasks.

Contribution

It presents a novel heterogeneous 3D network and a large multi-domain dataset for effective transfer learning in 3D medical image analysis.

Findings

Med3D accelerates training convergence by 2-10 times.

Pre-trained Med3D models improve accuracy by 3-20%.

Achieved near state-of-the-art results on LiTS challenge.

Abstract

The performance on deep learning is significantly affected by volume of training data. Models pre-trained from massive dataset such as ImageNet become a powerful weapon for speeding up training convergence and improving accuracy. Similarly, models based on large dataset are important for the development of deep learning in 3D medical images. However, it is extremely challenging to build a sufficiently large dataset due to difficulty of data acquisition and annotation in 3D medical imaging. We aggregate the dataset from several medical challenges to build 3DSeg-8 dataset with diverse modalities, target organs, and pathologies. To extract general medical three-dimension (3D) features, we design a heterogeneous 3D network called Med3D to co-train multi-domain 3DSeg-8 so as to make a series of pre-trained models. We transfer Med3D pre-trained models to lung segmentation in LIDC dataset, pulmonary nodule classification in LIDC dataset and liver segmentation on LiTS challenge. Experiments show that the Med3D can accelerate the training convergence speed of target 3D medical tasks 2 times compared with model pre-trained on Kinetics dataset, and 10 times compared with training from scratch as well as improve accuracy ranging from 3% to 20%. Transferring our Med3D model on state-the-of-art DenseASPP segmentation network, in case of single model, we achieve 94.6\% Dice coefficient which approaches the result of top-ranged algorithms on the LiTS challenge.