Analyzing Multimodal Features of Spontaneous Voice Assistant Commands for Mild Cognitive Impairment Detection

TL;DR

This study explores using spontaneous voice commands from older adults, combined with multimodal features, to detect mild cognitive impairment with high accuracy, highlighting the potential of in-home voice data for early diagnosis.

Contribution

It introduces a novel command-generation task and multimodal fusion models that improve MCI detection accuracy over traditional command-reading tasks.

Findings

Command-generation task achieves 82% classification accuracy.

Generated commands show stronger correlation with memory and attention.

Multimodal features enhance MCI detection performance.

Abstract

Mild cognitive impairment (MCI) is a major public health concern due to its high risk of progressing to dementia. This study investigates the potential of detecting MCI with spontaneous voice assistant (VA) commands from 35 older adults in a controlled setting. Specifically, a command-generation task is designed with pre-defined intents for participants to freely generate commands that are more associated with cognitive ability than read commands. We develop MCI classification and regression models with audio, textual, intent, and multimodal fusion features. We find the command-generation task outperforms the command-reading task with an average classification accuracy of 82%, achieved by leveraging multimodal fusion features. In addition, generated commands correlate more strongly with memory and attention subdomains than read commands. Our results confirm the effectiveness of the command-generation task and imply the promise of using longitudinal in-home commands for MCI detection.