Towards Robust Assessment of Pathological Voices via Combined Low-Level Descriptors and Foundation Model Representations

TL;DR

This paper presents VOQANet and VOQANet+, deep learning models that combine foundation model embeddings and low-level acoustic features to improve objective assessment of pathological voices, outperforming traditional methods and showing robustness in noisy conditions.

Contribution

Introduction of VOQANet+ that integrates foundation model embeddings with low-level descriptors for enhanced, robust voice quality assessment.

Findings

VOQANet outperforms baseline models in RMSE and correlation.

Sentence-level inputs improve accuracy over vowel-level.

VOQANet+ maintains performance under noisy conditions.

Abstract

Perceptual voice quality assessment plays a vital role in diagnosing and monitoring voice disorders. Traditional methods, such as the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) and the Grade, Roughness, Breathiness, Asthenia, and Strain (GRBAS) scales, rely on expert raters and are prone to inter-rater variability, emphasizing the need for objective solutions. This study introduces the Voice Quality Assessment Network (VOQANet), a deep learning framework that employs an attention mechanism and Speech Foundation Model (SFM) embeddings to extract high-level features. To further enhance performance, we propose VOQANet+, which integrates self-supervised SFM embeddings with low-level acoustic descriptors-namely jitter, shimmer, and harmonics-to-noise ratio (HNR). Unlike previous approaches that focus solely on vowel-based phonation (PVQD-A), our models are evaluated on both vowel-level and sentence-level speech (PVQD-S) to assess generalizability. Experimental results demonstrate that sentence-based inputs yield higher accuracy, particularly at the patient level. Overall, VOQANet consistently outperforms baseline models in terms of root mean squared error (RMSE) and Pearson correlation coefficient across CAPE-V and GRBAS dimensions, with VOQANet+ achieving even greater performance gains. Additionally, VOQANet+ maintains consistent performance under noisy conditions, suggesting enhanced robustness for real-world and telehealth applications. This work highlights the value of combining SFM embeddings with low-level features for accurate and robust pathological voice assessment.