# Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities

## TL;DR

This paper demonstrates that a single convolutional neural network can effectively perform multiple medical image segmentation tasks across different modalities, matching the performance of specialized models.

## Contribution

It introduces a multi-task CNN approach capable of segmenting various tissues and structures across different imaging modalities with high accuracy.

## Key findings

- Single CNN achieves performance comparable to task-specific models.
- Multi-task training enables modality and structure recognition.
- Potential for unified clinical segmentation systems.

## Abstract

Automatic segmentation of medical images is an important task for many clinical applications. In practice, a wide range of anatomical structures are visualised using different imaging modalities. In this paper, we investigate whether a single convolutional neural network (CNN) can be trained to perform different segmentation tasks. A single CNN is trained to segment six tissues in MR brain images, the pectoral muscle in MR breast images, and the coronary arteries in cardiac CTA. The CNN therefore learns to identify the imaging modality, the visualised anatomical structures, and the tissue classes. For each of the three tasks (brain MRI, breast MRI and cardiac CTA), this combined training procedure resulted in a segmentation performance equivalent to that of a CNN trained specifically for that task, demonstrating the high capacity of CNN architectures. Hence, a single system could be used in clinical practice to automatically perform diverse segmentation tasks without task-specific training.