Multimodal Assessment of Speech Impairment in ALS Using Audio-Visual and Machine Learning Approaches

TL;DR

This paper explores combining audio-visual data and machine learning to objectively assess speech impairment in ALS patients, aiming to improve early detection and monitoring beyond subjective clinical evaluations.

Contribution

It introduces a multimodal machine learning approach that improves speech impairment assessment accuracy using audio-visual features in ALS patients.

Findings

Best model achieved RMSE of 0.93, indicating high prediction accuracy.

Multimodal features outperform unimodal approaches.

Method offers potential for objective, accessible ALS speech assessment.

Abstract

The analysis of speech in individuals with amyotrophic lateral sclerosis is a powerful tool to support clinicians in the assessment of bulbar dysfunction. However, current methods used in clinical practice consist of subjective evaluations or expensive instrumentation. This study investigates different approaches combining audio-visual analysis and machine learning to predict the speech impairment evaluation performed by clinicians. Using a small dataset of acoustic and kinematic features extracted from audio and video recordings of speech tasks, we trained and tested some regression models. The best performance was achieved using the extreme boosting machine regressor with multimodal features, which resulted in a root mean squared error of 0.93 on a scale ranging from 5 to 25. Results suggest that integrating audio-video analysis enhances speech impairment assessment, providing an objective tool for early detection and monitoring of bulbar dysfunction, also in home settings.