Synergy-Based Evaluation of Hand Motor Function in Object Handling Using Virtual and Mixed Realities

TL;DR

This paper presents a new system using virtual and mixed realities to objectively evaluate hand motor function through synergy-based analysis.

Contribution

The novel system uses the Leap Motion Controller and HoloLens 2 for synergy-based evaluation of hand movements without glove-type devices.

Findings

Synergy-based analysis detects hand movement abnormalities not captured by traditional metrics.

The system enables more objective and detailed evaluations of finger motor function.

It shows potential for personalized therapy and early detection of motor impairments.

Abstract

This study introduces a novel system for evaluating hand motor function through synergy-based analysis during object manipulation in virtual and mixed-reality environments. Conventional assessments of hand function are often subjective, relying on visual observation by therapists or patient-reported outcomes. To address these limitations, we developed a system that utilizes the leap motion controller (LMC) to capture finger motion data without the constraints of glove-type devices. Spatial synergies were extracted using principal component analysis (PCA) and Varimax rotation, providing insights into finger motor coordination with the sparse decomposition. Additionally, we incorporated the HoloLens 2 to create a mixed-reality object manipulation task that enhances spatial awareness for the user, improving natural interaction with virtual objects. Our results demonstrate that synergy-based analysis allows for the systematic detection of hand movement abnormalities that are not captured through traditional task performance metrics. This system demonstrates promise in advancing rehabilitation by enabling more objective and detailed evaluations of finger motor function, facilitating personalized therapy, and potentially contributing to the early detection of motor impairments in the future.

Figures