Reconstructing Speech Stimuli From Human Auditory Cortex Activity Using a WaveNet Approach

TL;DR

This study demonstrates that a WaveNet model can reconstruct speech stimuli from intracranial recordings of the human auditory cortex, revealing phoneme-level tuning properties and advancing understanding of speech processing.

Contribution

The paper introduces a WaveNet-based approach capable of reconstructing speech from limited intracranial data and analyzes cortical phonetic tuning properties.

Findings

WaveNet successfully reconstructs speech from STG recordings.

Electrode-specific phoneme-level tuning observed.

Supports role of pSTG in phonetic speech representation.

Abstract

The superior temporal gyrus (STG) region of cortex critically contributes to speech recognition. In this work, we show that a proposed WaveNet, with limited available data, is able to reconstruct speech stimuli from STG intracranial recordings. We further investigate the impulse response of the fitted model for each recording electrode and observe phoneme level temporospectral tuning properties for the recorded area of cortex. This discovery is consistent with previous studies implicating the posterior STG (pSTG) in a phonetic representation of speech and provides detailed acoustic features that certain electrode sites possibly extract during speech recognition.