Towards Temporally Explainable Dysarthric Speech Clarity Assessment

TL;DR

This paper proposes a three-stage, explainable framework for assessing dysarthric speech clarity, including scoring, localization, and classification, using pretrained ASR models to aid speech therapy.

Contribution

It introduces a novel, temporally explainable assessment framework for dysarthric speech that leverages pretrained ASR models and provides actionable feedback.

Findings

Pretrained ASR models can effectively evaluate dysarthric speech clarity.

The framework offers clinically relevant insights for speech therapy.

Automated mispronunciation localization and classification are feasible.

Abstract

Dysarthria, a motor speech disorder, affects intelligibility and requires targeted interventions for effective communication. In this work, we investigate automated mispronunciation feedback by collecting a dysarthric speech dataset from six speakers reading two passages, annotated by a speech therapist with temporal markers and mispronunciation descriptions. We design a three-stage framework for explainable mispronunciation evaluation: (1) overall clarity scoring, (2) mispronunciation localization, and (3) mispronunciation type classification. We systematically analyze pretrained Automatic Speech Recognition (ASR) models in each stage, assessing their effectiveness in dysarthric speech evaluation (Code available at: https://github.com/augmented-human-lab/interspeech25_speechtherapy, Supplementary webpage: https://apps.ahlab.org/interspeech25_speechtherapy/). Our findings offer clinically relevant insights for automating actionable feedback for pronunciation assessment, which could enable independent practice for patients and help therapists deliver more effective interventions.