A Speech Intelligibility Enhancement Model based on Canonical Correlation and Deep Learning for Hearing-Assistive Technologies

TL;DR

This paper introduces a novel deep learning speech enhancement model that uses a canonical correlation-based loss function to improve speech intelligibility in noisy environments, outperforming traditional methods in unseen conditions.

Contribution

It presents the first integration of canonical correlation in an intelligibility-oriented loss function for speech enhancement, enhancing generalization and robustness.

Findings

Outperforms conventional STOI-based models in objective measures.

Achieves better subjective speech intelligibility scores.

Effective with unseen speakers and noise conditions.

Abstract

Current deep learning (DL) based approaches to speech intelligibility enhancement in noisy environments are generally trained to minimise the distance between clean and enhanced speech features. These often result in improved speech quality however they suffer from a lack of generalisation and may not deliver the required speech intelligibility in everyday noisy situations. In an attempt to address these challenges, researchers have explored intelligibility-oriented (I-O) loss functions to train DL approaches for robust speech enhancement (SE). In this paper, we formulate a novel canonical correlation-based I-O loss function to more effectively train DL algorithms. Specifically, we present a fully convolutional SE model that uses a modified canonical-correlation based short-time objective intelligibility (CC-STOI) metric as a training cost function. To the best of our knowledge, this is the first work that exploits the integration of canonical correlation in an I-O based loss function for SE. Comparative experimental results demonstrate that our proposed CC-STOI based SE framework outperforms DL models trained with conventional STOI and distance-based loss functions, in terms of both standard objective and subjective evaluation measures when dealing with unseen speakers and noises.