Mono-Modalizing Extremely Heterogeneous Multi-Modal Medical Image Registration

TL;DR

This paper introduces M2M-Reg, a novel framework for multi-modal deformable image registration that uses mono-modal similarity metrics and cyclic training to improve alignment of highly heterogeneous medical images like PET, FA, MRI, and CT.

Contribution

M2M-Reg is a new framework that enables effective multi-modal registration using only mono-modal similarity, addressing the challenge of heterogeneity in medical imaging modalities.

Findings

Achieves up to 2x higher DSC on ADNI dataset for PET-MRI and FA-MRI registration.

Effectively handles highly heterogeneous modalities with improved accuracy.

Extends to semi-supervised setting without needing ground-truth transformations.

Abstract

In clinical practice, imaging modalities with functional characteristics, such as positron emission tomography (PET) and fractional anisotropy (FA), are often aligned with a structural reference (e.g., MRI, CT) for accurate interpretation or group analysis, necessitating multi-modal deformable image registration (DIR). However, due to the extreme heterogeneity of these modalities compared to standard structural scans, conventional unsupervised DIR methods struggle to learn reliable spatial mappings and often distort images. We find that the similarity metrics guiding these models fail to capture alignment between highly disparate modalities. To address this, we propose M2M-Reg (Multi-to-Mono Registration), a novel framework that trains multi-modal DIR models using only mono-modal similarity while preserving the established architectural paradigm for seamless integration into existing models. We also introduce GradCyCon, a regularizer that leverages M2M-Reg's cyclic training scheme to promote diffeomorphism. Furthermore, our framework naturally extends to a semi-supervised setting, integrating pre-aligned and unaligned pairs only, without requiring ground-truth transformations or segmentation masks. Experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset demonstrate that M2M-Reg achieves up to 2x higher DSC than prior methods for PET-MRI and FA-MRI registration, highlighting its effectiveness in handling highly heterogeneous multi-modal DIR. Our code is available at https://github.com/MICV-yonsei/M2M-Reg.