Bridging the gap between paired and unpaired medical image translation

TL;DR

This paper introduces modified pix2pix models for unpaired medical image translation between CT and MR scans, achieving improved realism and alignment without requiring paired data or extensive annotations.

Contribution

The authors develop and validate new pix2pix variants that effectively translate between CT and MR images using unpaired data, outperforming existing models in quality metrics.

Findings

Modified pix2pix models outperform baseline methods in FID and KID scores.

The models produce more realistic and well-aligned CT and MR images.

Unpaired data suffices for high-quality medical image translation.

Abstract

Medical image translation has the potential to reduce the imaging workload, by removing the need to capture some sequences, and to reduce the annotation burden for developing machine learning methods. GANs have been used successfully to translate images from one domain to another, such as MR to CT. At present, paired data (registered MR and CT images) or extra supervision (e.g. segmentation masks) is needed to learn good translation models. Registering multiple modalities or annotating structures within each of them is a tedious and laborious task. Thus, there is a need to develop improved translation methods for unpaired data. Here, we introduce modified pix2pix models for tasks CT$\rightarrow$MR and MR$\rightarrow$CT, trained with unpaired CT and MR data, and MRCAT pairs generated from the MR scans. The proposed modifications utilize the paired MR and MRCAT images to ensure good alignment between input and translated images, and unpaired CT images ensure the MR$\rightarrow$CT model produces realistic-looking CT and CT$\rightarrow$MR model works well with real CT as input. The proposed pix2pix variants outperform baseline pix2pix, pix2pixHD and CycleGAN in terms of FID and KID, and generate more realistic looking CT and MR translations.