Augmented reality for upper limb rehabilitation: real-time kinematic feedback with HoloLens 2

TL;DR

This paper presents a HoloLens 2-based augmented reality system that provides real-time 3D feedback for upper limb rehabilitation, improving movement precision and user engagement in therapeutic exercises.

Contribution

It introduces a novel AR application with holographic trajectories and color-coded feedback, enhancing kinematic performance and usability for arm rehabilitation.

Findings

Significant improvement in movement accuracy with feedback system

Positive clinician evaluations of usability and acceptability

Encourages further testing with neurological disorder patients

Abstract

Exoskeletons for rehabilitation can help enhance motor recovery in individuals suffering from neurological disorders. Precision in movement execution, especially in arm rehabilitation, is crucial to prevent maladaptive plasticity. However, current exoskeletons, while providing arm support, often lack the necessary 3D feedback capabilities to show how well rehabilitation exercises are being performed. This reduces therapist acceptance and patients' performance. Augmented Reality technologies offer promising solutions for feedback and gaming systems in rehabilitation. In this work, we leverage HoloLens 2 with its advanced hand-tracking system to develop an application for personalized rehabilitation. Our application generates custom holographic trajectories based on existing databases or therapists' demonstrations, represented as 3D tunnels. Such trajectories can be superimposed on the real training environment. They serve as a guide to the users and, thanks to colour-coded real-time feedback, indicate their performance. To assess the efficacy of the application in improving kinematic precision, we tested it with 15 healthy subjects. Comparing user tracking capabilities with and without the use of our feedback system in executing 4 different exercises, we observed significant differences, demonstrating that our application leads to improved kinematic performance. 12 clinicians tested our system and positively evaluated its usability (System Usability Scale score of 67.7) and acceptability (4.4 out of 5 in the 'Willingness to Use' category in the relative Technology Acceptance Model). The results from the tests on healthy participants and the feedback from clinicians encourage further exploration of our framework, to verify its potential in supporting arm rehabilitation for individuals with neurological disorders.