EEG to fMRI Synthesis: Is Deep Learning a candidate?

TL;DR

This paper explores the potential of deep learning methods to synthesize fMRI brain images from EEG data, demonstrating feasibility and highlighting future improvements for accessible brain monitoring.

Contribution

It is the first comprehensive study applying neural processing principles to map EEG to fMRI data using various state-of-the-art models.

Findings

EEG to fMRI mapping is feasible with current machine learning techniques.

Autoencoders, GANs, and Pairwise Learning are effective for synthesis.

The approach can improve brain imaging accessibility and portability.

Abstract

Advances on signal, image and video generation underly major breakthroughs on generative medical imaging tasks, including Brain Image Synthesis. Still, the extent to which functional Magnetic Ressonance Imaging (fMRI) can be mapped from the brain electrophysiology remains largely unexplored. This work provides the first comprehensive view on how to use state-of-the-art principles from Neural Processing to synthesize fMRI data from electroencephalographic (EEG) data. Given the distinct spatiotemporal nature of haemodynamic and electrophysiological signals, this problem is formulated as the task of learning a mapping function between multivariate time series with highly dissimilar structures. A comparison of state-of-the-art synthesis approaches, including Autoencoders, Generative Adversarial Networks and Pairwise Learning, is undertaken. Results highlight the feasibility of EEG to fMRI brain image mappings, pinpointing the role of current advances in Machine Learning and showing the relevance of upcoming contributions to further improve performance. EEG to fMRI synthesis offers a way to enhance and augment brain image data, and guarantee access to more affordable, portable and long-lasting protocols of brain activity monitoring. The code used in this manuscript is available in Github and the datasets are open source.