Condition-Invariant fMRI Decoding of Speech Intelligibility with Deep State Space Model

TL;DR

This paper introduces a deep state space model to decode speech intelligibility from fMRI signals, demonstrating condition-invariant neural coding across diverse listening environments, and surpassing classical methods in accuracy.

Contribution

The study presents the first neural decoding approach that generalizes across different acoustic conditions using a novel deep state space model.

Findings

Decodes speech intelligibility across diverse acoustic conditions.

Identifies condition-invariant neural codes in brain regions.

Outperforms classical decoding approaches.

Abstract

Clarifying the neural basis of speech intelligibility is critical for computational neuroscience and digital speech processing. Recent neuroimaging studies have shown that intelligibility modulates cortical activity beyond simple acoustics, primarily in the superior temporal and inferior frontal gyri. However, previous studies have been largely confined to clean speech, leaving it unclear whether the brain employs condition-invariant neural codes across diverse listening environments. To address this gap, we propose a novel architecture built upon a deep state space model for decoding intelligibility from fMRI signals, specifically tailored to their high-dimensional temporal structure. We present the first attempt to decode intelligibility across acoustically distinct conditions, showing our method significantly outperforms classical approaches. Furthermore, region-wise analysis highlights contributions from auditory, frontal, and parietal regions, and cross-condition transfer indicates the presence of condition-invariant neural codes, thereby advancing understanding of abstract linguistic representations in the brain.