WhiSQA: Non-Intrusive Speech Quality Prediction Using Whisper Encoder Features

TL;DR

This paper introduces WhiSQA, a non-intrusive speech quality predictor leveraging Whisper encoder features, outperforming existing metrics in correlation with human ratings and demonstrating strong domain adaptation.

Contribution

The paper presents a novel speech quality prediction method using Whisper encoder features, achieving superior correlation with human ratings and better domain adaptation than existing metrics.

Findings

Higher correlation with human MOS ratings than recent approaches

Significantly better domain adaptation compared to DNSMOS

Robust performance across multiple test sets

Abstract

There has been significant research effort developing neural-network-based predictors of SQ in recent years. While a primary objective has been to develop non-intrusive, i.e.~reference-free, metrics to assess the performance of SE systems, recent work has also investigated the direct inference of neural SQ predictors within the loss function of downstream speech tasks. To aid in the training of SQ predictors, several large datasets of audio with corresponding human labels of quality have been created. Recent work in this area has shown that speech representations derived from large unsupervised or semi-supervised foundational speech models are useful input feature representations for neural SQ prediction. In this work, a novel and robust SQ predictor is proposed based on feature representations extracted from an ASR model, found to be a powerful input feature for the SQ prediction task. The proposed system achieves higher correlation with human MOS ratings than recent approaches on all NISQA test sets and shows significantly better domain adaption compared to the commonly used DNSMOS metric.