A Multimodal In Vitro Diagnostic Method for Parkinson's Disease Combining Facial Expressions and Behavioral Gait Data

TL;DR

This paper introduces a new multimodal diagnostic approach for Parkinson's disease that combines facial expressions and gait data, utilizing lightweight deep learning for improved accuracy and mobile deployment.

Contribution

It presents the first large-scale multimodal PD dataset and a novel deep learning model for integrated analysis of facial and gait data.

Findings

Enhanced diagnostic accuracy demonstrated in experiments

Successful deployment on mobile devices

Largest multimodal PD dataset established

Abstract

Parkinson's disease (PD), characterized by its incurable nature, rapid progression, and severe disability, poses significant challenges to the lives of patients and their families. Given the aging population, the need for early detection of PD is increasing. In vitro diagnosis has garnered attention due to its non-invasive nature and low cost. However, existing methods present several challenges: 1) limited training data for facial expression diagnosis; 2) specialized equipment and acquisition environments required for gait diagnosis, resulting in poor generalizability; 3) the risk of misdiagnosis or missed diagnosis when relying on a single modality. To address these issues, we propose a novel multimodal in vitro diagnostic method for PD, leveraging facial expressions and behavioral gait. Our method employs a lightweight deep learning model for feature extraction and fusion, aimed at improving diagnostic accuracy and facilitating deployment on mobile devices. Furthermore, we have established the largest multimodal PD dataset in collaboration with a hospital and conducted extensive experiments to validate the effectiveness of our proposed method.