Organ At Risk Segmentation with Multiple Modality

TL;DR

This paper introduces a multi-modality approach for organ at risk segmentation using a GAN to synthesize MR images from CT scans, improving segmentation accuracy in nasopharyngeal cancer cases.

Contribution

It proposes a novel method combining GAN-based image synthesis with instance segmentation to leverage multiple imaging modalities for better OAR segmentation.

Findings

Enhanced segmentation accuracy with multi-modality data

Successful synthesis of MR images from CT using GAN

Extended segmentation to include both organs and tumors

Abstract

With the development of image segmentation in computer vision, biomedical image segmentation have achieved remarkable progress on brain tumor segmentation and Organ At Risk (OAR) segmentation. However, most of the research only uses single modality such as Computed Tomography (CT) scans while in real world scenario doctors often use multiple modalities to get more accurate result. To better leverage different modalities, we have collected a large dataset consists of 136 cases with CT and MR images which diagnosed with nasopharyngeal cancer. In this paper, we propose to use Generative Adversarial Network to perform CT to MR transformation to synthesize MR images instead of aligning two modalities. The synthesized MR can be jointly trained with CT to achieve better performance. In addition, we use instance segmentation model to extend the OAR segmentation task to segment both organs and tumor region. The collected dataset will be made public soon.