Cross-modality Knowledge Transfer for Prostate Segmentation from CT Scans

TL;DR

This paper introduces a CycleGAN-based method to generate synthetic CT images from prostate MR scans, enabling accurate prostate segmentation in CT without requiring annotated CT data.

Contribution

The study presents a novel approach for cross-modality knowledge transfer using CycleGAN to improve CT prostate segmentation from unpaired MR data.

Findings

Synthetic CT images enable accurate prostate segmentation.

Segmentation results are comparable to radiologist annotations.

Method addresses lack of annotated data in medical imaging.

Abstract

Creating large scale high-quality annotations is a known challenge in medical imaging. In this work, based on the CycleGAN algorithm, we propose leveraging annotations from one modality to be useful in other modalities. More specifically, the proposed algorithm creates highly realistic synthetic CT images (SynCT) from prostate MR images using unpaired data sets. By using SynCT images (without segmentation labels) and MR images (with segmentation labels available), we have trained a deep segmentation network for precise delineation of prostate from real CT scans. For the generator in our CycleGAN, the cycle consistency term is used to guarantee that SynCT shares the identical manually-drawn, high-quality masks originally delineated on MR images. Further, we introduce a cost function based on structural similarity index (SSIM) to improve the anatomical similarity between real and synthetic images. For segmentation followed by the SynCT generation from CycleGAN, automatic delineation is achieved through a 2.5D Residual U-Net. Quantitative evaluation demonstrates comparable segmentation results between our SynCT and radiologist drawn masks for real CT images, solving an important problem in medical image segmentation field when ground truth annotations are not available for the modality of interest.