Generation of annotated multimodal ground truth datasets for abdominal medical image registration

TL;DR

This paper introduces a method to generate annotated multimodal 4D medical image datasets using CycleGAN, enabling improved registration and segmentation tasks despite limited real annotated data.

Contribution

The authors develop a novel framework that creates synthetic multimodal 4D datasets with ground truth annotations, enhancing medical image registration and segmentation capabilities.

Findings

Synthetic data closely matches real data in intensity and noise characteristics.

Generated datasets are inherently co-registered across modalities.

Framework improves registration parameter optimization using organ masks.

Abstract

Sparsity of annotated data is a major limitation in medical image processing tasks such as registration. Registered multimodal image data are essential for the diagnosis of medical conditions and the success of interventional medical procedures. To overcome the shortage of data, we present a method that allows the generation of annotated multimodal 4D datasets. We use a CycleGAN network architecture to generate multimodal synthetic data from the 4D extended cardiac-torso (XCAT) phantom and real patient data. Organ masks are provided by the XCAT phantom, therefore the generated dataset can serve as ground truth for image segmentation and registration. Realistic simulation of respiration and heartbeat is possible within the XCAT framework. To underline the usability as a registration ground truth, a proof of principle registration is performed. Compared to real patient data, the synthetic data showed good agreement regarding the image voxel intensity distribution and the noise characteristics. The generated T1-weighted magnetic resonance imaging (MRI), computed tomography (CT), and cone beam CT (CBCT) images are inherently co-registered. Thus, the synthetic dataset allowed us to optimize registration parameters of a multimodal non-rigid registration, utilizing liver organ masks for evaluation. Our proposed framework provides not only annotated but also multimodal synthetic data which can serve as a ground truth for various tasks in medical imaging processing. We demonstrated the applicability of synthetic data for the development of multimodal medical image registration algorithms.