Structurally Consistent MRI Colorization using Cross-modal Fusion Learning

TL;DR

This paper introduces a novel cross-modal fusion architecture for MRI colorization that enhances interpretability by transferring Cryosection color information without requiring precise registration or MRI segmentation.

Contribution

The proposed architecture uniquely fuses Cryosection segmentation semantics and employs feature compression to achieve stable, organ-specific MRI colorization without needing MRI segmentation or registration.

Findings

Outperforms existing methods in quantitative metrics

Provides more realistic and organ-specific colorization

Does not require MRI segmentation or registration

Abstract

Medical image colorization can greatly enhance the interpretability of the underlying imaging modality and provide insights into human anatomy. The objective of medical image colorization is to transfer a diverse spectrum of colors distributed across human anatomy from Cryosection data to source MRI data while retaining the structures of the MRI. To achieve this, we propose a novel architecture for structurally consistent color transfer to the source MRI data. Our architecture fuses segmentation semantics of Cryosection images for stable contextual colorization of various organs in MRI images. For colorization, we neither require precise registration between MRI and Cryosection images, nor segmentation of MRI images. Additionally, our architecture incorporates a feature compression-and-activation mechanism to capture organ-level global information and suppress noise, enabling the distinction of organ-specific data in MRI scans for more accurate and realistic organ-specific colorization. Our experiments demonstrate that our architecture surpasses the existing methods and yields better quantitative and qualitative results.