A Convolutional Framework for Mapping Imagined Auditory MEG into Listened Brain Responses

TL;DR

This study introduces a neural network framework that effectively maps imagined auditory responses to listened responses in MEG data, advancing brain-computer interface potential for speech and music.

Contribution

We developed a convolutional neural network with subject-specific calibration that outperforms traditional models in mapping imagined to perceived brain responses.

Findings

Imagined and perceived responses contain condition-specific information.

The CNN model outperforms null models with higher correlation scores.

Limited generalization across individual subjects at the single-subject level.

Abstract

Decoding imagined speech engages complex neural processes that are difficult to interpret due to uncertainty in timing and the limited availability of imagined-response datasets. In this study, we present a Magnetoencephalography (MEG) dataset collected from trained musicians as they imagined and listened to musical and poetic stimuli. We show that both imagined and perceived brain responses contain consistent, condition-specific information. Using a sliding-window ridge regression model, we first mapped imagined responses to listened responses at the single-subject level, but found limited generalization across subjects. At the group level, we developed an encoder-decoder convolutional neural network with a subject-specific calibration layer that produced stable and generalizable mappings. The CNN consistently outperformed the null model, yielding significantly higher correlations between predicted and true listened responses for nearly all held-out subjects. Our findings demonstrate that imagined neural activity can be transformed into perception-like responses, providing a foundation for future brain-computer interface applications involving imagined speech and music.