Optimal Transport-based Adaptation in Dysarthric Speech Tasks

TL;DR

This paper introduces an optimal transport-based multi-source domain adaptation method for dysarthric speech, significantly improving detection accuracy, command recognition, and dysarthria diagnosis by leveraging speaker similarity measures.

Contribution

It proposes MSDA-WDJOT, a novel optimal transport approach for dysarthric speech adaptation, outperforming existing models in detection, recognition, and diagnosis tasks.

Findings

0.9% improvement in dysarthria detection accuracy

16% reduction in command error rate

95% accuracy in dysarthria diagnosis

Abstract

In many real-world applications, the mismatch between distributions of training data (source) and test data (target) significantly degrades the performance of machine learning algorithms. In speech data, causes of this mismatch include different acoustic environments or speaker characteristics. In this paper, we address this issue in the challenging context of dysarthric speech, by multi-source domain/speaker adaptation (MSDA/MSSA). Specifically, we propose the use of an optimal-transport based approach, called MSDA via Weighted Joint Optimal Transport (MSDA-WDJOT). We confront the mismatch problem in dysarthria detection for which the proposed approach outperforms both the Baseline and the state-of-the-art MSDA models, improving the detection accuracy of 0.9% over the best competitor method. We then employ MSDA-WJDOT for dysarthric speaker adaptation in command speech recognition. This provides a Command Error Rate relative reduction of 16% and 7% over the baseline and the best competitor model, respectively. Interestingly, MSDA-WJDOT provides a similarity score between the source and the target, i.e. between speakers in this case. We leverage this similarity measure to define a Dysarthric and Healthy score of the target speaker and diagnose the dysarthria with an accuracy of 95%.