Natural Typing Recognition via Surface Electromyography

TL;DR

This paper introduces a deep learning system that accurately recognizes typed text from surface electromyography signals captured via a computer keyboard, achieving over 90% accuracy and enabling real-time gesture recognition.

Contribution

It presents the largest sEMG gesture dataset and a novel temporal-focused neural network architecture for high-accuracy, real-time typing recognition from muscle signals.

Findings

Achieved over 90% character-level accuracy in text reconstruction.

Demonstrated robustness of the system after synthetic downgrades of data resolution.

Showed potential for real-time gesture recognition applications.

Abstract

By using a computer keyboard as a finger recording device, we construct the largest existing dataset for gesture recognition via surface electromyography (sEMG), and use deep learning to achieve over 90% character-level accuracy on reconstructing typed text entirely from measured muscle potentials. We prioritize the temporal structure of the EMG signal instead of the spatial structure of the electrode layout, using network architectures inspired by those used for real-time spoken language transcription. Our architecture recognizes the rapid movements of natural computer typing, which occur at irregular intervals and often overlap in time. The extensive size of our dataset also allows us to study gesture recognition after synthetically downgrading the spatial or temporal resolution, showing the system capabilities necessary for real-time gesture recognition.