Resolving Domain Shift For Representations Of Speech In Non-Invasive Brain Recordings

TL;DR

This paper introduces a novel adversarial domain adaptation framework to improve speech decoding from non-invasive MEG neuroimaging data across multiple datasets, addressing the challenge of domain shift and demographic effects.

Contribution

It is the first to apply feature-level deep learning harmonization for MEG data, enhancing model generalization across datasets and revealing demographic influences.

Findings

Improved speech decoding performance across datasets.

Demonstrated age significantly affects neuroimaging data.

Provided open-source implementation for broader use.

Abstract

Machine learning techniques have enabled researchers to leverage neuroimaging data to decode speech from brain activity, with some amazing recent successes achieved by applications built using invasive devices. However, research requiring surgical implants has a number of practical limitations. Non-invasive neuroimaging techniques provide an alternative but come with their own set of challenges, the limited scale of individual studies being among them. Without the ability to pool the recordings from different non-invasive studies, data on the order of magnitude needed to leverage deep learning techniques to their full potential remains out of reach. In this work, we focus on non-invasive data collected using magnetoencephalography (MEG). We leverage two different, leading speech decoding models to investigate how an adversarial domain adaptation framework augments their ability to generalize across datasets. We successfully improve the performance of both models when training across multiple datasets. To the best of our knowledge, this study is the first ever application of feature-level, deep learning based harmonization for MEG neuroimaging data. Our analysis additionally offers further evidence of the impact of demographic features on neuroimaging data, demonstrating that participant age strongly affects how machine learning models solve speech decoding tasks using MEG data. Lastly, in the course of this study we produce a new open-source implementation of one of these models to the benefit of the broader scientific community.