Early soft and flexible fusion of EEG and fMRI via tensor decompositions

TL;DR

This paper introduces a novel tensor-based fusion approach for EEG and fMRI data that leverages soft and flexible coupling, outperforming traditional ICA and hard coupling methods in analyzing brain activity.

Contribution

It pioneers the use of tensor models with soft coupling for joint EEG-fMRI analysis, addressing multi-way data structure and variability in brain imaging.

Findings

Tensor methods outperform ICA in brain data analysis.

Soft and flexible coupling improves fusion when hard assumptions are violated.

PARAFAC2 effectively handles EEG ERP variability.

Abstract

Data fusion refers to the joint analysis of multiple datasets which provide complementary views of the same task. In this preprint, the problem of jointly analyzing electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) data is considered. Jointly analyzing EEG and fMRI measurements is highly beneficial for studying brain function because these modalities have complementary spatiotemporal resolution: EEG offers good temporal resolution while fMRI is better in its spatial resolution. The fusion methods reported so far ignore the underlying multi-way nature of the data in at least one of the modalities and/or rely on very strong assumptions about the relation of the two datasets. In this preprint, these two points are addressed by adopting for the first time tensor models in the two modalities while also exploring double coupled tensor decompositions and by following soft and flexible coupling approaches to implement the multi-modal analysis. To cope with the Event Related Potential (ERP) variability in EEG, the PARAFAC2 model is adopted. The results obtained are compared against those of parallel Independent Component Analysis (ICA) and hard coupling alternatives in both simulated and real data. Our results confirm the superiority of tensorial methods over methods based on ICA. In scenarios that do not meet the assumptions underlying hard coupling, the advantage of soft and flexible coupled decompositions is clearly demonstrated.