Tensor Analysis and Fusion of Multimodal Brain Images

TL;DR

This paper introduces tensor-based methods and Markov-Penrose diagrams for multimodal brain imaging data fusion, enabling improved analysis of brain structures and networks across different neuroimaging modalities.

Contribution

It presents a novel tensor analysis framework and visualization tools for multimodal neuroimaging data fusion, including the first tensor regression approach to Granger causal analysis of brain networks.

Findings

Tensor regression enables atomic decomposition of brain networks.

Analysis of EEG and fMRI demonstrates method effectiveness.

Potential applications extend to other scientific domains.

Abstract

Current high-throughput data acquisition technologies probe dynamical systems with different imaging modalities, generating massive data sets at different spatial and temporal resolutions posing challenging problems in multimodal data fusion. A case in point is the attempt to parse out the brain structures and networks that underpin human cognitive processes by analysis of different neuroimaging modalities (functional MRI, EEG, NIRS etc.). We emphasize that the multimodal, multi-scale nature of neuroimaging data is well reflected by a multi-way (tensor) structure where the underlying processes can be summarized by a relatively small number of components or "atoms". We introduce Markov-Penrose diagrams - an integration of Bayesian DAG and tensor network notation in order to analyze these models. These diagrams not only clarify matrix and tensor EEG and fMRI time/frequency analysis and inverse problems, but also help understand multimodal fusion via Multiway Partial Least Squares and Coupled Matrix-Tensor Factorization. We show here, for the first time, that Granger causal analysis of brain networks is a tensor regression problem, thus allowing the atomic decomposition of brain networks. Analysis of EEG and fMRI recordings shows the potential of the methods and suggests their use in other scientific domains.