Brain Image Synthesis with Unsupervised Multivariate Canonical CSC$\ell_4$Net

TL;DR

This paper introduces CSCℓ4Net, a novel neural network that synthesizes missing neuroimaging modalities from source data by learning cross-modal features, enabling effective reconstruction despite data heterogeneity and limited modality availability.

Contribution

CSCℓ4Net is a new deep learning framework that unifies intra- and inter-modal features for neuroimaging data synthesis, incorporating Riemannian manifold regularization and efficient optimization.

Findings

Outperforms state-of-the-art methods on multiple datasets.

Robustly reconstructs missing neuroimaging modalities.

Effectively handles high-dimensional, heterogeneous data.

Abstract

Recent advances in neuroscience have highlighted the effectiveness of multi-modal medical data for investigating certain pathologies and understanding human cognition. However, obtaining full sets of different modalities is limited by various factors, such as long acquisition times, high examination costs and artifact suppression. In addition, the complexity, high dimensionality and heterogeneity of neuroimaging data remains another key challenge in leveraging existing randomized scans effectively, as data of the same modality is often measured differently by different machines. There is a clear need to go beyond the traditional imaging-dependent process and synthesize anatomically specific target-modality data from a source input. In this paper, we propose to learn dedicated features that cross both intre- and intra-modal variations using a novel CSC$\ell_4$Net. Through an initial unification of intra-modal data in the feature maps and multivariate canonical adaptation, CSC$\ell_4$Net facilitates feature-level mutual transformation. The positive definite Riemannian manifold-penalized data fidelity term further enables CSC$\ell_4$Net to reconstruct missing measurements according to transformed features. Finally, the maximization $\ell_4$-norm boils down to a computationally efficient optimization problem. Extensive experiments validate the ability and robustness of our CSC$\ell_4$Net compared to the state-of-the-art methods on multiple datasets.