CAtCh: Cognitive Assessment through Cookie Thief

TL;DR

This study evaluates speech-based machine learning methods for predicting broader cognitive impairment, finding multimodal approaches and acoustic features outperform linguistic ones, with implications for early detection of cognitive decline.

Contribution

It introduces and compares multimodal speech analysis methods for cognitive impairment prediction, highlighting the effectiveness of acoustic features over linguistic features.

Findings

Multimodal methods outperform unimodal ones in CI prediction.

Acoustic features related to affect and prosody are more effective than linguistic features.

Acoustic approaches outperform linguistics-based methods in predicting cognitive impairment.

Abstract

Several machine learning algorithms have been developed for the prediction of Alzheimer's disease and related dementia (ADRD) from spontaneous speech. However, none of these algorithms have been translated for the prediction of broader cognitive impairment (CI), which in some cases is a precursor and risk factor of ADRD. In this paper, we evaluated several speech-based open-source methods originally proposed for the prediction of ADRD, as well as methods from multimodal sentiment analysis for the task of predicting CI from patient audio recordings. Results demonstrated that multimodal methods outperformed unimodal ones for CI prediction, and that acoustics-based approaches performed better than linguistics-based ones. Specifically, interpretable acoustic features relating to affect and prosody were found to significantly outperform BERT-based linguistic features and interpretable linguistic features, respectively. All the code developed for this study is available at https://github.com/JTColonel/catch.