Cognitive Insights Across Languages: Enhancing Multimodal Interview Analysis

TL;DR

This paper introduces a multimodal model that analyzes audio and text from clinical interviews to predict cognitive decline and differentiate languages, aiding early diagnosis of cognitive impairments.

Contribution

It presents a novel multimodal architecture that integrates audio and text features for improved cognitive impairment prediction across languages.

Findings

Effective differentiation between languages in interviews

Accurate prediction of Mild Cognitive Impairment

Robust multimodal feature integration

Abstract

Cognitive decline is a natural process that occurs as individuals age. Early diagnosis of anomalous decline is crucial for initiating professional treatment that can enhance the quality of life of those affected. To address this issue, we propose a multimodal model capable of predicting Mild Cognitive Impairment and cognitive scores. The TAUKADIAL dataset is used to conduct the evaluation, which comprises audio recordings of clinical interviews. The proposed model demonstrates the ability to transcribe and differentiate between languages used in the interviews. Subsequently, the model extracts audio and text features, combining them into a multimodal architecture to achieve robust and generalized results. Our approach involves in-depth research to implement various features obtained from the proposed modalities.