Latent neural source recovery via transcoding of simultaneous EEG-fMRI

TL;DR

This paper introduces a symmetric transcoding method using cyclic convolutional neural networks to infer and recover neural source spaces from simultaneous EEG-fMRI data without prior knowledge of specific response functions, enabling cost-effective neuroimaging.

Contribution

The paper presents a novel cyclic convolutional transcoder that symmetrically maps EEG to fMRI and vice versa, learning these mappings directly from data without prior knowledge.

Findings

High-quality transcoding between EEG and fMRI demonstrated on real data

Effective recovery of neural source spaces from unseen data

Potential for low-cost neuroimaging by generating fMRI images from EEG

Abstract

Simultaneous EEG-fMRI is a multi-modal neuroimaging technique that provides complementary spatial and temporal resolution for inferring a latent source space of neural activity. In this paper we address this inference problem within the framework of transcoding -- mapping from a specific encoding (modality) to a decoding (the latent source space) and then encoding the latent source space to the other modality. Specifically, we develop a symmetric method consisting of a cyclic convolutional transcoder that transcodes EEG to fMRI and vice versa. Without any prior knowledge of either the hemodynamic response function or lead field matrix, the method exploits the temporal and spatial relationships between the modalities and latent source spaces to learn these mappings. We show, for real EEG-fMRI data, how well the modalities can be transcoded from one to another as well as the source spaces that are recovered, all on unseen data. In addition to enabling a new way to symmetrically infer a latent source space, the method can also be seen as low-cost computational neuroimaging -- i.e. generating an 'expensive' fMRI BOLD image from 'low cost' EEG data.