CoCoLIT: ControlNet-Conditioned Latent Image Translation for MRI to Amyloid PET Synthesis

TL;DR

CoCoLIT is a diffusion-based latent generative framework that improves MRI to amyloid PET translation for Alzheimer's screening by introducing novel loss functions, stabilization techniques, and conditioning methods, outperforming existing approaches.

Contribution

The paper introduces CoCoLIT, a novel MRI-to-PET translation model utilizing ControlNet conditioning, a new loss function, and stabilization analysis, advancing the state-of-the-art in neuroimaging synthesis.

Findings

Significantly outperforms existing methods on image quality metrics.

Achieves +10.5% and +23.7% improvements in amyloid-positivity classification accuracy.

Demonstrates effective latent space modeling for complex neuroimaging data.

Abstract

Synthesizing amyloid PET scans from the more widely available and accessible structural MRI modality offers a promising, cost-effective approach for large-scale Alzheimer's Disease (AD) screening. This is motivated by evidence that, while MRI does not directly detect amyloid pathology, it may nonetheless encode information correlated with amyloid deposition that can be uncovered through advanced modeling. However, the high dimensionality and structural complexity of 3D neuroimaging data pose significant challenges for existing MRI-to-PET translation methods. Modeling the cross-modality relationship in a lower-dimensional latent space can simplify the learning task and enable more effective translation. As such, we present CoCoLIT (ControlNet-Conditioned Latent Image Translation), a diffusion-based latent generative framework that incorporates three main innovations: (1) a novel Weighted Image Space Loss (WISL) that improves latent representation learning and synthesis quality; (2) a theoretical and empirical analysis of Latent Average Stabilization (LAS), an existing technique used in similar generative models to enhance inference consistency; and (3) the introduction of ControlNet-based conditioning for MRI-to-PET translation. We evaluate CoCoLIT's performance on publicly available datasets and find that our model significantly outperforms state-of-the-art methods on both image-based and amyloid-related metrics. Notably, in amyloid-positivity classification, CoCoLIT outperforms the second-best method with improvements of +10.5% on the internal dataset and +23.7% on the external dataset. The code and models of our approach are available at https://github.com/brAIn-science/CoCoLIT.