Cog-TiPRO: Iterative Prompt Refinement with LLMs to Detect Cognitive Decline via Longitudinal Voice Assistant Commands

TL;DR

This study presents Cog-TiPRO, a novel framework using large language models and acoustic analysis to detect cognitive decline from longitudinal voice commands, achieving significant accuracy improvements over baselines.

Contribution

We introduce Cog-TiPRO, an innovative iterative prompt refinement framework combining linguistic and acoustic features for early cognitive decline detection from voice data.

Findings

Achieved 73.80% accuracy in MCI detection

Outperformed baseline by 27.13%

Identified unique linguistic features of cognitive decline

Abstract

Early detection of cognitive decline is crucial for enabling interventions that can slow neurodegenerative disease progression. Traditional diagnostic approaches rely on labor-intensive clinical assessments, which are impractical for frequent monitoring. Our pilot study investigates voice assistant systems (VAS) as non-invasive tools for detecting cognitive decline through longitudinal analysis of speech patterns in voice commands. Over an 18-month period, we collected voice commands from 35 older adults, with 15 participants providing daily at-home VAS interactions. To address the challenges of analyzing these short, unstructured and noisy commands, we propose Cog-TiPRO, a framework that combines (1) LLM-driven iterative prompt refinement for linguistic feature extraction, (2) HuBERT-based acoustic feature extraction, and (3) transformer-based temporal modeling. Using iTransformer, our approach achieves 73.80% accuracy and 72.67% F1-score in detecting MCI, outperforming its baseline by 27.13%. Through our LLM approach, we identify linguistic features that uniquely characterize everyday command usage patterns in individuals experiencing cognitive decline.