Exploring Motion Boundaries in an End-to-End Network for Vision-based Parkinson's Severity Assessment

TL;DR

This paper introduces an end-to-end deep learning approach utilizing motion boundaries and attention mechanisms to assess Parkinson's disease severity from video data, achieving promising accuracy on hand and gait tasks.

Contribution

The novel integration of motion boundaries and temporal attention in an Inflated 3D CNN for PD severity assessment from videos.

Findings

Achieved 72.3% accuracy on hand movement tasks.

Achieved 77.1% accuracy on gait tasks.

Demonstrated effectiveness of motion boundaries as an input modality.

Abstract

Evaluating neurological disorders such as Parkinson's disease (PD) is a challenging task that requires the assessment of several motor and non-motor functions. In this paper, we present an end-to-end deep learning framework to measure PD severity in two important components, hand movement and gait, of the Unified Parkinson's Disease Rating Scale (UPDRS). Our method leverages on an Inflated 3D CNN trained by a temporal segment framework to learn spatial and long temporal structure in video data. We also deploy a temporal attention mechanism to boost the performance of our model. Further, motion boundaries are explored as an extra input modality to assist in obfuscating the effects of camera motion for better movement assessment. We ablate the effects of different data modalities on the accuracy of the proposed network and compare with other popular architectures. We evaluate our proposed method on a dataset of 25 PD patients, obtaining 72.3% and 77.1% top-1 accuracy on hand movement and gait tasks respectively.