Prosthetic Hand Manipulation System Based on EMG and Eye Tracking Powered by the Neuromorphic Processor AltAi

TL;DR

This paper introduces a neuromorphic control system for prosthetic hands that combines EMG and eye tracking, achieving high accuracy and low power consumption for real-time, safe, and reliable prosthesis manipulation.

Contribution

It presents a novel neuromorphic architecture deploying a spiking neural network on AltAi, integrating EMG and gaze data for improved prosthetic control.

Findings

Comparable accuracy to GPU-based models in EMG classification

Achieves roughly 95% accuracy with context-aware gesture recognition

Operates in a sub-watt power regime suitable for wearable devices

Abstract

This paper presents a novel neuromorphic control architecture for upper-limb prostheses that combines surface electromyography (sEMG) with gaze-guided computer vision. The system uses a spiking neural network deployed on the neuromorphic processor AltAi to classify EMG patterns in real time while an eye-tracking headset and scene camera identify the object within the user's focus. In our prototype, the same EMG recognition model that was originally developed for a conventional GPU is deployed as a spiking network on AltAi, achieving comparable accuracy while operating in a sub-watt power regime, which enables a lightweight, wearable implementation. For six distinct functional gestures recorded from upper-limb amputees, the system achieves robust recognition performance comparable to state-of-the-art myoelectric interfaces. When the vision pipeline restricts the decision space to three context-appropriate gestures for the currently viewed object, recognition accuracy increases to roughly 95% while excluding unsafe, object-inappropriate grasps. These results indicate that the proposed neuromorphic, context-aware controller can provide energy-efficient and reliable prosthesis control and has the potential to improve safety and usability in everyday activities for people with upper-limb amputation.