Feature Importance across Domains for Improving Non-Intrusive Speech Intelligibility Prediction in Hearing Aids

TL;DR

This paper introduces FiDo, a method that improves non-intrusive speech intelligibility prediction in hearing aids by estimating feature importance across domains, leading to more accurate assessments.

Contribution

The paper proposes a novel feature importance estimation method across spectral, time-domain, and latent features, enhancing neural speech assessment models in hearing aids.

Findings

RMSE reduced by 7.62% with FiDo

Achieved 3.98% relative RMSE reduction over best challenge system

Validated FiDo's effectiveness in neural speech assessment

Abstract

Given the critical role of non-intrusive speech intelligibility assessment in hearing aids (HA), this paper enhances its performance by introducing Feature Importance across Domains (FiDo). We estimate feature importance on spectral and time-domain acoustic features as well as latent representations of Whisper. Importance weights are calculated per frame, and based on these weights, features are projected into new spaces, allowing the model to focus on important areas early. Next, feature concatenation is performed to combine the features before the assessment module processes them. Experimental results show that when FiDo is incorporated into the improved multi-branched speech intelligibility model MBI-Net+, RMSE can be reduced by 7.62% (from 26.10 to 24.11). MBI-Net+ with FiDo also achieves a relative RMSE reduction of 3.98% compared to the best system in the 2023 Clarity Prediction Challenge. These results validate FiDo's effectiveness in enhancing neural speech assessment in HA.