SilentWear: an Ultra-Low Power Wearable System for EMG-based Silent Speech Recognition

TL;DR

SilentWear is a wearable textile-based EMG system for silent speech recognition that operates efficiently on-device, offering high accuracy, robustness to repositioning, and long battery life for practical human-computer interaction.

Contribution

This work introduces SilentWear, a fully wearable EMG-based speech recognition system with on-device processing, novel fine-tuning strategies, and demonstrated real-time performance on a commercial microcontroller.

Findings

Achieves 84.8% accuracy for vocalized speech and 77.5% for silent speech.

Demonstrates over 10% accuracy improvement with incremental fine-tuning.

Operates continuously for more than 27 hours with low energy consumption.

Abstract

Detecting speech from biosignals is gaining increasing attention due to the potential to develop human-computer interfaces that are noise-robust, privacy-preserving, and scalable for both clinical applications and daily use. However, most existing approaches remain limited by insufficient wearability and the lack of edge-processing capabilities, which are essential for minimally obtrusive, responsive, and private assistive technologies. In this work, we present SilentWear, a fully wearable, textile-based neck interface for EMG signal acquisition and processing. Powered by BioGAP-Ultra, the system enables end-to-end data acquisition from 14 differential channels and on-device speech recognition. SilentWear is coupled with SpeechNet, a lightweight 15k-parameter CNN architecture specifically tailored for EMG-based speech decoding, achieving an average cross-validated accuracy of 84.8$\pm$4.6% and 77.5$\pm$6.6% for vocalized and silent speech, respectively, over eight representative human-machine interaction commands collected over multiple days. We evaluate robustness to repositioning induced by multi-day use. In an inter-session setting, the system achieves average accuracies of 71.1$\pm$8.3% and 59.3\pm2.2% for vocalized and silent speech, respectively. To mitigate performance degradation due to repositioning, we propose an incremental fine-tuning strategy, demonstrating more than 10% accuracy recovery with less than 10 minutes of additional user data. Finally, we demonstrate end-to-end real-time on-device speech recognition on a commercial multi-core microcontroller unit (MCU), achieving an energy consumption of 63.9$\mu$J per inference with a latency of 2.47 ms. With a total power consumption of 20.5mW for acquisition, inference, and wireless transmission of results, SilentWear enables continuous operation for more than 27 hours.