Deep Recurrent Encoder: A scalable end-to-end network to model brain signals

TL;DR

This paper introduces a deep recurrent encoder architecture that significantly improves the prediction of brain responses from MEG data during reading tasks, offering better modeling of nonlinear brain dynamics compared to traditional linear methods.

Contribution

The paper presents a novel end-to-end deep learning model for brain signal prediction that outperforms linear methods and includes an interpretability approach to understand neural responses.

Findings

Three-fold improvement over linear methods in predicting MEG responses.

Successful recovery of expected neural responses to word features.

Scalable approach applicable to large brain datasets.

Abstract

Understanding how the brain responds to sensory inputs is challenging: brain recordings are partial, noisy, and high dimensional; they vary across sessions and subjects and they capture highly nonlinear dynamics. These challenges have led the community to develop a variety of preprocessing and analytical (almost exclusively linear) methods, each designed to tackle one of these issues. Instead, we propose to address these challenges through a specific end-to-end deep learning architecture, trained to predict the brain responses of multiple subjects at once. We successfully test this approach on a large cohort of magnetoencephalography (MEG) recordings acquired during a one-hour reading task. Our Deep Recurrent Encoding (DRE) architecture reliably predicts MEG responses to words with a three-fold improvement over classic linear methods. To overcome the notorious issue of interpretability of deep learning, we describe a simple variable importance analysis. When applied to DRE, this method recovers the expected evoked responses to word length and word frequency. The quantitative improvement of the present deep learning approach paves the way to better understand the nonlinear dynamics of brain activity from large datasets.