Exploiting Hidden Representations from a DNN-based Speech Recogniser for Speech Intelligibility Prediction in Hearing-impaired Listeners

TL;DR

This paper introduces a novel approach that uses hidden representations from a DNN-based speech recognizer to improve the prediction of speech intelligibility for hearing-impaired listeners, outperforming traditional methods.

Contribution

The work leverages deep neural network hidden layers as features for intelligibility prediction, providing a new perspective beyond conventional acoustic feature-based methods.

Findings

Proposed method outperforms STOI-based binaural measure in prediction accuracy.

Using DNN hidden representations improves speech intelligibility prediction.

Experiment results validate the effectiveness of the approach on hearing aid data.

Abstract

An accurate objective speech intelligibility prediction algorithms is of great interest for many applications such as speech enhancement for hearing aids. Most algorithms measures the signal-to-noise ratios or correlations between the acoustic features of clean reference signals and degraded signals. However, these hand-picked acoustic features are usually not explicitly correlated with recognition. Meanwhile, deep neural network (DNN) based automatic speech recogniser (ASR) is approaching human performance in some speech recognition tasks. This work leverages the hidden representations from DNN-based ASR as features for speech intelligibility prediction in hearing-impaired listeners. The experiments based on a hearing aid intelligibility database show that the proposed method could make better prediction than a widely used short-time objective intelligibility (STOI) based binaural measure.