Complex Mapping between Neural Response Frequency and Linguistic Units in Natural Speech

TL;DR

This study investigates how neural responses track linguistic units in natural speech, revealing that frequency-based methods cannot distinctly separate responses to different units like syllables or words.

Contribution

It demonstrates that frequency-domain analysis alone cannot reliably differentiate neural responses to various linguistic units in natural speech.

Findings

Neural responses to phones, syllables, and words correlate with speech envelope at 3-6 Hz and below 1 Hz.

1-Hz correlation mainly arises from pauses in speech.

Frequency-based decomposition is insufficient to distinguish responses to different linguistic units.

Abstract

When listening to connected speech, human brain can extract multiple levels of linguistic units, such as syllables, words, and sentences. It has been hypothesized that the time scale of cortical activity encoding each linguistic unit is commensurate with the time scale of that linguistic unit in speech. Evidence for the hypothesis originally comes from studies using the frequency-tagging paradigm that presents each linguistic unit at a constant rate, and more recently extends to studies on natural speech. For natural speech, it is sometimes assumed that neural encoding of different levels of linguistic units is captured by the neural response tracking speech envelope in different frequency bands (e.g., around 1 Hz for phrases, around 2 Hz for words, and around 4 Hz for syllables). Here, we analyze the coherence between speech envelope and idealized responses, each of which tracks a single level of linguistic unit. Four units, i.e., phones, syllables, words, and sentences, are separately considered. It is shown that the idealized phone-, syllable-, and word-tracking responses all correlate with the speech envelope both around 3-6 Hz and below ~1 Hz. Further analyses reveal that the 1-Hz correlation mainly originates from the pauses in connected speech. The results here suggest that a simple frequency-domain decomposition of envelope-tracking activity cannot separate the neural responses to different linguistic units in natural speech.