Speaker-Independent Dysarthria Severity Classification using Self-Supervised Transformers and Multi-Task Learning

TL;DR

This paper introduces a transformer-based, speaker-independent framework called SALR for automatic dysarthria severity classification from raw speech, outperforming traditional methods and establishing new benchmarks in clinical speech assessment.

Contribution

The study presents a novel SALR transformer framework with multi-task and contrastive learning for speaker-independent dysarthria severity classification, improving accuracy and robustness over prior approaches.

Findings

Achieved 70.48% accuracy and 59.23% F1 score on the Universal Access Speech dataset.

Exceeded previous SVM benchmark by 16.58%.

Visualized latent space to demonstrate reduced speaker-specific cues.

Abstract

Dysarthria, a condition resulting from impaired control of the speech muscles due to neurological disorders, significantly impacts the communication and quality of life of patients. The condition's complexity, human scoring and varied presentations make its assessment and management challenging. This study presents a transformer-based framework for automatically assessing dysarthria severity from raw speech data. It can offer an objective, repeatable, accessible, standardised and cost-effective and compared to traditional methods requiring human expert assessors. We develop a transformer framework, called Speaker-Agnostic Latent Regularisation (SALR), incorporating a multi-task learning objective and contrastive learning for speaker-independent multi-class dysarthria severity classification. The multi-task framework is designed to reduce reliance on speaker-specific characteristics and address the intrinsic intra-class variability of dysarthric speech. We evaluated on the Universal Access Speech dataset using leave-one-speaker-out cross-validation, our model demonstrated superior performance over traditional machine learning approaches, with an accuracy of $70.48\%$ and an F1 score of $59.23\%$. Our SALR model also exceeded the previous benchmark for AI-based classification, which used support vector machines, by $16.58\%$. We open the black box of our model by visualising the latent space where we can observe how the model substantially reduces speaker-specific cues and amplifies task-specific ones, thereby showing its robustness. In conclusion, SALR establishes a new benchmark in speaker-independent multi-class dysarthria severity classification using generative AI. The potential implications of our findings for broader clinical applications in automated dysarthria severity assessments.