Inferring latent neural sources via deep transcoding of simultaneously acquired EEG and fMRI

TL;DR

This paper introduces a data-driven, symmetric transcoding method using cyclic convolutional neural networks to infer latent neural source spaces from simultaneous EEG and fMRI data, enabling cost-effective neuroimaging.

Contribution

It develops a novel cyclic convolutional transcoder that maps between EEG and fMRI modalities without prior knowledge, facilitating symmetric inference of neural source spaces.

Findings

Successfully transcoded EEG to fMRI and vice versa in simulated data

Recovered meaningful latent neural source spaces from real data

Demonstrated potential for low-cost neuroimaging using EEG data

Abstract

Simultaneous EEG-fMRI is a multi-modal neuroimaging technique that provides complementary spatial and temporal resolution. Challenging has been developing principled and interpretable approaches for fusing the modalities, specifically approaches enabling inference of latent source spaces representative of neural activity. In this paper, we address this inference problem within the framework of transcoding -- mapping from a specific encoding (modality) to a decoding (the latent source space) and then encoding the latent source space to the other modality. Specifically, we develop a symmetric method consisting of a cyclic convolutional transcoder that transcodes EEG to fMRI and vice versa. Without any prior knowledge of either the hemodynamic response function or lead field matrix, the complete data-driven method exploits the temporal and spatial relationships between the modalities and latent source spaces to learn these mappings. We quantify, for both the simulated and real EEG-fMRI data, how well the modalities can be transcoded from one to another as well as the source spaces that are recovered, all evaluated on unseen data. In addition to enabling a new way to symmetrically infer a latent source space, the method can also be seen as low-cost computational neuroimaging -- i.e. generating an 'expensive' fMRI BOLD image from 'low cost' EEG data.