A Multi-decoder Neural Tracking Method for Accurately Predicting Speech Intelligibility

TL;DR

This paper presents a multi-decoder neural tracking method that accurately predicts speech reception thresholds from EEG data, enabling objective speech intelligibility assessment especially for populations unable to perform behavioral tests.

Contribution

The study introduces a novel multi-decoder approach that aggregates diverse neural tracking features to predict speech intelligibility, reducing data collection time without sacrificing accuracy.

Findings

Predicted SRTs significantly correlated with behavioral measures (r=0.647).

All prediction differences were under 1 dB, matching behavioral test precision.

Pretrained decoders reduced EEG recording time from 29 to 15 minutes.

Abstract

Objective: EEG-based methods can predict speech intelligibility, but their accuracy and robustness lag behind behavioral tests, which typically show test-retest differences under 1 dB. We introduce the multi-decoder method to predict speech reception thresholds (SRTs) from EEG recordings, enabling objective assessment for populations unable to perform behavioral tests; such as those with disorders of consciousness or during hearing aid fitting. Approach: The method aggregates data from hundreds of decoders, each trained on different speech features and EEG preprocessing setups to quantify neural tracking (NT) of speech signals. Using data from 39 participants (ages 18-24), we recorded 29 minutes of EEG per person while they listened to speech at six signal-to-noise ratios and a quiet story. NT values were combined into a high-dimensional feature vector per subject, and a support vector regression model was trained to predict SRTs from these vectors. Main Result: Predictions correlated significantly with behavioral SRTs (r = 0.647, p < 0.001; NRMSE = 0.19), with all differences under 1 dB. SHAP analysis showed theta/delta bands and early lags had slightly greater influence. Using pretrained subject-independent decoders reduced required EEG data collection to 15 minutes (3 minutes of story, 12 minutes across six SNR conditions) without losing accuracy.