i-GSI: A Fast and Reliable Grasp-type Switching Interface based on Augmented Reality and Eye-tracking

TL;DR

This paper introduces i-GSI, an augmented reality and eye-tracking based interface for rapid, reliable grasp-type switching in prosthetic hands, outperforming traditional methods in speed and accuracy.

Contribution

The paper presents a novel AR and eye-tracking integrated grasp-type switching interface (i-GSI) that operates in real-time on HoloLens2, improving efficiency and success rates.

Findings

Achieved a switching time of 0.84 seconds in healthy subjects.

Attained a 99.0% success rate in grasp-type switching.

Demonstrated effective use with a patient with limb deficiency, with 0.78 seconds switching time.

Abstract

The control of multi-fingered dexterous prosthetics hand remains challenging due to the lack of an intuitive and efficient Grasp-type Switching Interface (GSI). We propose a new GSI (i-GSI) t hat integrates the manifold power of eye-tracking and augmentced reality technologies to solve this problem. It runs entirely in a HoloLens2 helmet, where users can glance at icons on the holographic interface to switch between six daily grasp types quickly. Compared to traditional GSIs (FSM-based, PR-based, and mobile APP-based), i-GSI achieved the best results in the experiment with eight healthy subjects, achieving a switching time of 0.84 s, a switching success rate of 99.0%, and learning efficiency of 93.50%. By verifying on one patient with a congenital upper limb deficiency, i-GSI achieved an equivalent great outcome as on healthy people, with a switching time of 0.78 s and switching success rate of 100%. The new i-GSI, as a standalone module, can be combined with traditional proportional myoelectric control to form a hybrid-controlled prosthetic system that can help patients accomplish dexterous operations in various daily-life activities.