Generalized Multi-Task Learning from Substantially Unlabeled Multi-Source Medical Image Data

TL;DR

This paper introduces MultiMix, a semi-supervised multi-task learning model that jointly performs disease classification and anatomical segmentation in medical images, effectively utilizing unlabeled data and demonstrating strong generalization capabilities.

Contribution

The paper presents MultiMix, a novel semi-supervised multi-task learning framework with a saliency bridge for explainability, improving medical image analysis with limited labeled data.

Findings

Effective in classifying pneumonia and segmenting lungs in chest X-rays

Shows strong generalization in cross-domain evaluations

Utilizes unlabeled data to enhance learning performance

Abstract

Deep learning-based models, when trained in a fully-supervised manner, can be effective in performing complex image analysis tasks, although contingent upon the availability of large labeled datasets. Especially in the medical imaging domain, however, expert image annotation is expensive, time-consuming, and prone to variability. Semi-supervised learning from limited quantities of labeled data has shown promise as an alternative. Maximizing knowledge gains from copious unlabeled data benefits semi-supervised learning models. Moreover, learning multiple tasks within the same model further improves its generalizability. We propose MultiMix, a new multi-task learning model that jointly learns disease classification and anatomical segmentation in a semi-supervised manner, while preserving explainability through a novel saliency bridge between the two tasks. Our experiments with varying quantities of multi-source labeled data in the training sets confirm the effectiveness of MultiMix in the simultaneous classification of pneumonia and segmentation of the lungs in chest X-ray images. Moreover, both in-domain and cross-domain evaluations across these tasks further showcase the potential of our model to adapt to challenging generalization scenarios.