Bioformers: Embedding Transformers for Ultra-Low Power sEMG-based Gesture Recognition

TL;DR

This paper introduces Bioformers, ultra-small attention-based neural networks for sEMG gesture recognition, achieving state-of-the-art accuracy with significantly reduced memory, energy consumption, and latency, suitable for low-power embedded devices.

Contribution

The paper presents Bioformers, a novel family of compact attention-based models, and a new inter-subject pre-training method that enhances accuracy without increasing inference costs.

Findings

Bioformers reduce parameters and operations by 4.9X.

Achieve 3.39% accuracy improvement with pre-training.

Deploy on PULP MCU with 8X lower energy and latency.

Abstract

Human-machine interaction is gaining traction in rehabilitation tasks, such as controlling prosthetic hands or robotic arms. Gesture recognition exploiting surface electromyographic (sEMG) signals is one of the most promising approaches, given that sEMG signal acquisition is non-invasive and is directly related to muscle contraction. However, the analysis of these signals still presents many challenges since similar gestures result in similar muscle contractions. Thus the resulting signal shapes are almost identical, leading to low classification accuracy. To tackle this challenge, complex neural networks are employed, which require large memory footprints, consume relatively high energy and limit the maximum battery life of devices used for classification. This work addresses this problem with the introduction of the Bioformers. This new family of ultra-small attention-based architectures approaches state-of-the-art performance while reducing the number of parameters and operations of 4.9X. Additionally, by introducing a new inter-subjects pre-training, we improve the accuracy of our best Bioformer by 3.39%, matching state-of-the-art accuracy without any additional inference cost. Deploying our best performing Bioformer on a Parallel, Ultra-Low Power (PULP) microcontroller unit (MCU), the GreenWaves GAP8, we achieve an inference latency and energy of 2.72 ms and 0.14 mJ, respectively, 8.0X lower than the previous state-of-the-art neural network, while occupying just 94.2 kB of memory.