No Audiogram: Leveraging Existing Scores for Personalized Speech Intelligibility Prediction

TL;DR

This paper introduces SSIPNet, a deep learning model that predicts individual speech intelligibility using existing data, outperforming traditional audiogram-based methods even with limited support data.

Contribution

The paper presents a novel deep learning approach that leverages support samples to predict personalized speech intelligibility without relying on audiograms.

Findings

Outperforms audiogram-based predictions with few support samples

Effective on Clarity Prediction Challenge dataset

Introduces a new paradigm for personalized speech prediction

Abstract

Personalized speech intelligibility prediction is challenging. Previous approaches have mainly relied on audiograms, which are inherently limited in accuracy as they only capture a listener's hearing threshold for pure tones. Rather than incorporating additional listener features, we propose a novel approach that leverages an individual's existing intelligibility data to predict their performance on new audio. We introduce the Support Sample-Based Intelligibility Prediction Network (SSIPNet), a deep learning model that leverages speech foundation models to build a high-dimensional representation of a listener's speech recognition ability from multiple support (audio, score) pairs, enabling accurate predictions for unseen audio. Results on the Clarity Prediction Challenge dataset show that, even with a small number of support (audio, score) pairs, our method outperforms audiogram-based predictions. Our work presents a new paradigm for personalized speech intelligibility prediction.