Leveraging Digital Assessments and Speech Analytics for Monitoring Changes in Sleep and Cognition

TL;DR

This paper explores using digital assessments and speech analytics to track changes in sleep and cognition over time, aiming to improve early detection of cognitive decline.

Contribution

The novel contribution is integrating digital cognitive assessments with AI-powered speech analytics to monitor intraindividual and interindividual variability in sleep and cognition.

Findings

Digital assessments and speech analytics can track daily cognitive and sleep fluctuations.

Speech characteristics correlate with cognitive and sleep changes, detectable via ML algorithms.

Biometric data from actigraphy enhances models for characterizing cognitive states.

Abstract

Tracking everyday cognitive functioning, sleep, and mood, beyond a single timepoint of assessment, is essential for understanding the progression of cognitive impairment and improving early diagnosis and treatment of disorders like AD/ADRDs. In this talk, we will explore the integration of digital cognitive assessments and speech analytics, focusing on how these tools can track intraindividual and interindividual variability in sleep and cognitive performance over time. We will present our approach, developed in collaboration between IBM and McLean Hospital/Harvard Medical School, which combines traditional neuropsychological evaluations with cutting-edge speech analytics powered by AI, machine learning (ML), and large language models (LLMs). By leveraging digital cognitive assessments, we capture cognitive and sleep fluctuations through daily prompts and testing, allowing for a more dynamic understanding of cognitive health. In addition, speech data collected through open-ended prompts and summaries are analyzed using advanced ML algorithms, including Whisper from OpenAI for acoustic analysis, and chatGPT for result interpretation, to detect subtle changes in speech characteristics that correlate with cognitive and sleep changes. This talk will highlight how our tools track cognitive and sleep fluctuations over time, focusing on intraindividual variability—how an individual’s sleep impacts their cognitive changes from day to day—and interindividual variability—how these changes compare across different individuals. By integrating biometric data from actigraphy monitors, such as heart rate and activity levels, we will discuss how one can further refine models to better characterize cognitive states and identify how sleep can be used to detect early signs of cognitive decline.