Building medical image classifiers with very limited data using segmentation networks

TL;DR

This paper introduces a novel approach for medical image classification that leverages segmentation networks pre-trained on similar data, enabling effective learning from very limited datasets by focusing on shared morphological features.

Contribution

The study proposes a curriculum learning-inspired framework that uses segmentation networks to improve classification accuracy with small datasets, outperforming traditional transfer learning methods.

Findings

Achieved 82% accuracy on brain tumor classification with only 91 training samples.

Achieved 86% accuracy on cardiac classification with 108 training samples.

Outperformed ImageNet pre-trained and from-scratch classifiers in limited data scenarios.

Abstract

Deep learning has shown promising results in medical image analysis, however, the lack of very large annotated datasets confines its full potential. Although transfer learning with ImageNet pre-trained classification models can alleviate the problem, constrained image sizes and model complexities can lead to unnecessary increase in computational cost and decrease in performance. As many common morphological features are usually shared by different classification tasks of an organ, it is greatly beneficial if we can extract such features to improve classification with limited samples. Therefore, inspired by the idea of curriculum learning, we propose a strategy for building medical image classifiers using features from segmentation networks. By using a segmentation network pre-trained on similar data as the classification task, the machine can first learn the simpler shape and structural concepts before tackling the actual classification problem which usually involves more complicated concepts. Using our proposed framework on a 3D three-class brain tumor type classification problem, we achieved 82% accuracy on 191 testing samples with 91 training samples. When applying to a 2D nine-class cardiac semantic level classification problem, we achieved 86% accuracy on 263 testing samples with 108 training samples. Comparisons with ImageNet pre-trained classifiers and classifiers trained from scratch are presented.